给力的linux命令--parallel安装

TimiZheng

GNU parallel是并行执行任务的shell工具。在文章情景linux—shell如何实现多线程？中提到了它的一种用法，但这远远不够，它还有许多更强大的应用场景。但在此之前，需要先安装它。

下载

1. <font color="black" size="2">$ wget http://ftp.gnu.org/gnu/parallel/parallel-latest.tar.bz2
   
2. --2017-06-23 19:00:03--  http://ftp.gnu.org/gnu/parallel/parallel-latest.tar.bz2
   
3. Resolving ftp.gnu.org... 208.118.235.20, 2001:4830:134:3::b
   
4. Connecting to ftp.gnu.org|208.118.235.20|:80... connected.
   
5. HTTP request sent, awaiting response... 200 OK
   
6. Length: 1559659 (1.5M) [application/x-bzip2]
   
7. Saving to: “parallel-latest.tar.bz2”
   
8. 
   
9. 100%[=============================>] 1,559,659   4.46K/s   in 6m 19s  
   
10. 
    
11. 2017-06-23 19:06:23 (4.02 KB/s) - “parallel-latest.tar.bz2” saved [1559659/1559659]</font>

复制代码

查看下载到的文件：

1. $ ll
   
2. -rw-rw-r-- 1 j-tester j-tester 1559659 Jun 22 20:06 parallel-latest.tar.bz2

复制代码

解压下载的文件

1. tar -jvx -f parallel-latest.tar.bz2 -C ./
   
2. parallel-20170622/
   
3. parallel-20170622/COPYING
   
4. parallel-20170622/NEWS
   
5. parallel-20170622/configure
   
6. parallel-20170622/CITATION
   
7. parallel-20170622/missing
   
8. parallel-20170622/install-sh
   
9. parallel-20170622/src/
   
10. parallel-20170622/src/sem.pod
    
11. parallel-20170622/src/parset
    
12. parallel-20170622/src/parallel_tutorial.7
    
13. parallel-20170622/src/parallel_tutorial.pdf
    
14. parallel-20170622/src/parallel_alternatives.7
    
15. parallel-20170622/src/env_parallel.bash
    
16. parallel-20170622/src/parallel_alternatives.html
    
17. parallel-20170622/src/niceload.pdf
    
18. parallel-20170622/src/parset.pdf
    
19. parallel-20170622/src/parallel_design.pod
    
20. parallel-20170622/src/niceload.1
    
21. parallel-20170622/src/env_parallel
    
22. parallel-20170622/src/parset.pod
    
23. parallel-20170622/src/sem
    
24. parallel-20170622/src/env_parallel.html
    
25. parallel-20170622/src/env_parallel.pdf
    
26. parallel-20170622/src/niceload
    
27. parallel-20170622/src/parcat.pod
    
28. parallel-20170622/src/sql.pdf
    
29. parallel-20170622/src/parcat.texi
    
30. parallel-20170622/src/env_parallel.tcsh
    
31. parallel-20170622/src/sem.pdf
    
32. parallel-20170622/src/env_parallel.sh
    
33. parallel-20170622/src/niceload.html
    
34. parallel-20170622/src/parallel_design.7
    
35. parallel-20170622/src/env_parallel.ash
    
36. parallel-20170622/src/parallel_tutorial.texi
    
37. parallel-20170622/src/env_parallel.1
    
38. parallel-20170622/src/parallel.html
    
39. parallel-20170622/src/parallel_tutorial.pod
    
40. parallel-20170622/src/sql.1
    
41. parallel-20170622/src/parcat.html
    
42. parallel-20170622/src/parset.1
    
43. parallel-20170622/src/env_parallel.pod
    
44. parallel-20170622/src/parallel.pdf
    
45. parallel-20170622/src/env_parallel.texi
    
46. parallel-20170622/src/env_parallel.csh
    
47. parallel-20170622/src/niceload.pod
    
48. parallel-20170622/src/env_parallel.ksh
    
49. parallel-20170622/src/parcat
    
50. parallel-20170622/src/env_parallel.pdksh
    
51. parallel-20170622/src/parallel_design.pdf
    
52. parallel-20170622/src/env_parallel.zsh
    
53. parallel-20170622/src/parallel_alternatives.pod
    
54. parallel-20170622/src/parcat.pdf
    
55. parallel-20170622/src/Makefile.am
    
56. parallel-20170622/src/parcat.1
    
57. parallel-20170622/src/sem.texi
    
58. parallel-20170622/src/parallel_design.texi
    
59. parallel-20170622/src/parallel_alternatives.pdf
    
60. parallel-20170622/src/parallel
    
61. parallel-20170622/src/parset.texi
    
62. parallel-20170622/src/parallel_alternatives.texi
    
63. parallel-20170622/src/niceload.texi
    
64. parallel-20170622/src/parallel.texi
    
65. parallel-20170622/src/parallel.1
    
66. parallel-20170622/src/sql.html
    
67. parallel-20170622/src/sem.html
    
68. parallel-20170622/src/env_parallel.dash
    
69. parallel-20170622/src/parallel_design.html
    
70. parallel-20170622/src/env_parallel.fish
    
71. parallel-20170622/src/Makefile.in
    
72. parallel-20170622/src/sql.texi
    
73. parallel-20170622/src/sql
    
74. parallel-20170622/src/parallel.pod
    
75. parallel-20170622/src/parset.html
    
76. parallel-20170622/src/sem.1
    
77. parallel-20170622/src/parallel_tutorial.html
    
78. parallel-20170622/config.h.in
    
79. parallel-20170622/README
    
80. parallel-20170622/aclocal.m4
    
81. parallel-20170622/CREDITS
    
82. parallel-20170622/Makefile.am
    
83. parallel-20170622/configure.ac
    
84. parallel-20170622/Makefile.in

复制代码

进入解压后的文件夹

1. cd parallel-20170622/
   
2. total 388
   
3. -rw-rw-r-- 1 j-tester j-tester  28344 Jun 22 19:43 aclocal.m4
   
4. -rw-rw-r-- 1 j-tester j-tester    631 May  1 15:05 CITATION
   
5. -rw-rw-r-- 1 j-tester j-tester    625 Jun 22 19:43 config.h.in
   
6. -rwxrwxr-x 1 j-tester j-tester 110321 Jun 22 19:43 configure
   
7. -rw-rw-r-- 1 j-tester j-tester    442 Jun 22 19:35 configure.ac
   
8. -rw-rw-r-- 1 j-tester j-tester  35147 May  1 15:05 COPYING
   
9. -rw-rw-r-- 1 j-tester j-tester   1810 May  1 15:05 CREDITS
   
10. -rwxr-xr-x 1 j-tester j-tester  15155 Jun 22 19:43 install-sh
    
11. -rw-rw-r-- 1 j-tester j-tester   4733 May  1 15:05 Makefile.am
    
12. -rw-rw-r-- 1 j-tester j-tester  28687 Jun 22 19:43 Makefile.in
    
13. -rwxr-xr-x 1 j-tester j-tester   6872 Jun 22 19:43 missing
    
14. -rw-rw-r-- 1 j-tester j-tester 130719 Jun 22 19:45 NEWS
    
15. -rw-rw-r-- 1 j-tester j-tester   3525 Jun 22 19:35 README
    
16. drwxrwxr-x 2 j-tester j-tester   4096 Jun 22 19:45 src

复制代码

生成Makefile

1. $ ./configure
   
2. checking for a BSD-compatible install... /usr/bin/install -c
   
3. checking whether build environment is sane... yes
   
4. checking for a thread-safe mkdir -p... /bin/mkdir -p
   
5. checking for gawk... gawk
   
6. checking whether make sets $(MAKE)... yes
   
7. checking whether make supports nested variables... yes
   
8. checking whether ln -s works... yes
   
9. checking that generated files are newer than configure... done
   
10. configure: creating ./config.status
    
11. config.status: creating Makefile
    
12. config.status: creating src/Makefile
    
13. config.status: creating config.h

复制代码

编译

1. make -j4
   
2. make  all-recursive
   
3. make[1]: Entering directory `/home/j-tester/bin/multiple_process/parallel-20170622'
   
4. Making all in src
   
5. make[2]: Entering directory `/home/j-tester/bin/multiple_process/parallel-20170622/src'
   
6. make[2]: Nothing to be done for `all'.
   
7. make[2]: Leaving directory `/home/j-tester/bin/multiple_process/parallel-20170622/src'
   
8. make[2]: Entering directory `/home/j-tester/bin/multiple_process/parallel-20170622'
   
9. make[2]: Leaving directory `/home/j-tester/bin/multiple_process/parallel-20170622'
   
10. make[1]: Leaving directory `/home/j-tester/bin/multiple_process/parallel-20170622'
    
11. 
    

复制代码

来源：公众号软件测试技能站；作者：signjing（博为峰网校金牌讲师）

TimiZheng

安装

1. sudo make install -j 16
   
2. Making install in src
   
3. make[1]: Entering directory `/home/j-tester/bin/multiple_process/parallel-20170622/src'
   
4. make[2]: Entering directory `/home/j-tester/bin/multiple_process/parallel-20170622/src'
   
5. /bin/mkdir -p '/usr/local/bin'
   
6. /bin/mkdir -p '/usr/local/share/doc/parallel'
   
7. /bin/mkdir -p '/usr/local/share/man/man1'
   
8. /bin/mkdir -p '/usr/local/share/man/man7'
   
9. /usr/bin/install -c parallel sql niceload parcat env_parallel env_parallel.ash env_parallel.bash env_parallel.csh env_parallel.dash env_parallel.fish env_parallel.ksh env_parallel.pdksh env_parallel.sh env_parallel.tcsh env_parallel.zsh '/usr/local/bin'
   
10. /usr/bin/install -c -m 644 parallel.html env_parallel.html sem.html sql.html niceload.html parallel_tutorial.html parallel_design.html parallel_alternatives.html parcat.html parset.html parallel.texi env_parallel.texi sem.texi sql.texi niceload.texi parallel_tutorial.texi parallel_design.texi parallel_alternatives.texi parcat.texi parset.texi parallel.pdf env_parallel.pdf sem.pdf sql.pdf niceload.pdf parallel_tutorial.pdf parallel_design.pdf parallel_alternatives.pdf parcat.pdf parset.pdf '/usr/local/share/doc/parallel'
    
11. /usr/bin/install -c -m 644 parallel.1 env_parallel.1 sem.1 sql.1 niceload.1 parcat.1 parset.1 '/usr/local/share/man/man1'
    
12. /usr/bin/install -c -m 644 parallel_tutorial.7 parallel_design.7 parallel_alternatives.7 '/usr/local/share/man/man7'
    
13. make  install-exec-hook
    
14. make[3]: Entering directory `/home/j-tester/bin/multiple_process/parallel-20170622/src'
    
15. rm /usr/local/bin/sem || true
    
16. rm: cannot remove `/usr/local/bin/sem': No such file or directory
    
17. ln -s parallel /usr/local/bin/sem
    
18. make[3]: Leaving directory `/home/j-tester/bin/multiple_process/parallel-20170622/src'
    
19. make[2]: Leaving directory `/home/j-tester/bin/multiple_process/parallel-20170622/src'
    
20. make[1]: Leaving directory `/home/j-tester/bin/multiple_process/parallel-20170622/src'
    
21. make[1]: Entering directory `/home/j-tester/bin/multiple_process/parallel-20170622'
    
22. make[2]: Entering directory `/home/j-tester/bin/multiple_process/parallel-20170622'
    
23. make[2]: Nothing to be done for `install-exec-am'.
    
24. make[2]: Nothing to be done for `install-data-am'.
    
25. make[2]: Leaving directory `/home/j-tester/bin/multiple_process/parallel-20170622'
    
26. make[1]: Leaving directory `/home/j-tester/bin/multiple_process/parallel-20170622'

复制代码

TimiZheng

测试安装后的效果

1. whereis parallel
   
2. parallel: /usr/local/bin/parallel
   
3. $ parallel --help
   
4. Usage:
   
5. 
   
6. parallel [options] [command [arguments]] < list_of_arguments
   
7. parallel [options] [command [arguments]] (::: arguments|:::: argfile(s))...
   
8. cat ... | parallel --pipe [options] [command [arguments]]
   
9. 
   
10. -j n            Run n jobs in parallel
    
11. -k              Keep same order
    
12. -X              Multiple arguments with context replace
    
13. --colsep regexp Split input on regexp for positional replacements
    
14. {} {.} {/} {/.} {#} {%} {= perl code =} Replacement strings
    
15. {3} {3.} {3/} {3/.} {=3 perl code =}    Positional replacement strings
    
16. With --plus:    {} = {+/}/{/} = {.}.{+.} = {+/}/{/.}.{+.} = {..}.{+..} =
    
17.                 {+/}/{/..}.{+..} = {...}.{+...} = {+/}/{/...}.{+...}
    
18. 
    
19. -S sshlogin     Example: foo@server.example.com
    
20. --slf ..        Use ~/.parallel/sshloginfile as the list of sshlogins
    
21. --trc {}.bar    Shorthand for --transfer --return {}.bar --cleanup
    
22. --onall         Run the given command with argument on all sshlogins
    
23. --nonall        Run the given command with no arguments on all sshlogins
    
24. 
    
25. --pipe          Split stdin (standard input) to multiple jobs.
    
26. --recend str    Record end separator for --pipe.
    
27. --recstart str  Record start separator for --pipe.
    
28. 
    
29. See 'man parallel' for details
    
30. 
    
31. Academic tradition requires you to cite works you base your article on.
    
32. When using programs that use GNU Parallel to process data for publication
    
33. please cite:
    
34. 
    
35.   O. Tange (2011): GNU Parallel - The Command-Line Power Tool,
    
36.   ;login: The USENIX Magazine, February 2011:42-47.
    
37. 
    
38. This helps funding further development; AND IT WON'T COST YOU A CENT.
    
39. If you pay 10000 EUR you should feel free to use GNU Parallel without citing.

复制代码

参考链接
https://www.gnu.org/software/parallel/
情景linux—shell如何实现多线程？