The default behavior of the bash shell and the Linux operating system is to execute typed commands sequentially in serial order. So, when we type multiple commands on the terminal prompt and chain them using semicolons then the shell would first execute one command, wait for it to finish execution and then execute the next command and so on. This sequential or serial behavior is fine for most scenarios but sometimes we may have the need to execute commands in a parallel fashion. The general requirement for parallel command execution arises from the necessity to save time i.e. do more in less time. Currently, there are many automation tools and utilities available that allow users to perform tasks parallel on the same system or across different systems.
But what if we want to accomplish parallel task execution without installing any additional tools?
By starting multiple processes in background appending the ampersand character (“&”) to the end of your commands, we can simulate the startup of multiple processes in parallel on the command line. Although this approach is not perfect but could suffice when we need to fire numerous commands in parallel on the same or different machines. The main alternative to running a process in the foreground is to allow it to execute in the background. A background process is associated with the specific terminal that started it but does not block access to the shell. Instead, it executes in the background, leaving the user able to interact with the system while the command runs. Because of the way that a foreground process interacts with its terminal, there can be only a single foreground process for every terminal window. Because background processes return control to the shell immediately without waiting for the process to complete, many background processes can run at the same time.
In this article, we will demonstrate how to execute multiple programs or jobs simultaneously thereby giving the impression of parallel process/task execution.
Running tasks in parallel on the same system:
To demonstrate this, let’s execute the ping command and send ICMP ping packets to two hosts simultaneously. We’ll execute the following command.
[sahil@linuxnix:~] $ ping -c 2 google.com & ping -c2 linuxnix.com &  17422  17423 [sahil@linuxnix:~] $ PING google.com (220.127.116.11) 56(84) bytes of data. 64 bytes from dfw25s07-in-f14.1e100.net (18.104.22.168): icmp_seq=1 ttl=51 time=19.1 ms PING linuxnix.com (22.214.171.124) 56(84) bytes of data. 64 bytes from dfw25s07-in-f14.1e100.net (126.96.36.199): icmp_seq=2 ttl=51 time=19.0 ms --- google.com ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1021ms rtt min/avg/max/mdev = 19.091/19.111/19.131/0.020 ms 64 bytes from 188.8.131.52: icmp_seq=1 ttl=53 time=18.7 ms 64 bytes from 184.108.40.206: icmp_seq=2 ttl=53 time=18.8 ms --- linuxnix.com ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1263ms rtt min/avg/max/mdev = 18.753/18.777/18.801/0.024 ms - Done ping -c 2 google.com + Done ping -c2 linuxnix.com
Once we press the enter key after typing the commands, we first see the process IDs of the processes being generated by the execution of the commands. Next we see the output of the commands and finally, we get notified that the commands have completed execution.
Running tasks in parallel across different systems using SSH:
If we could execute tasks in parallel over SSH across different systems then that could really help us save a lot of time. In the below example, we demonstrate how to simultaneously start two SSH connections in the background thereby giving the impression of parallel command line execution.
[ssuri@uslinuxnix01:~] $ ssh uslinuxnix02 -q "uname -a;date;exit" >> abc.txt & ssh uslinuxnix05 -q "uname -a;date;exit" >> abc.txt &  21233  21234 [ssuri@uslinuxnix01:~] $ - Done ssh uslinuxnix02 -q "uname -a;date;exit" >> abc.txt + Done ssh uslinuxnix05 -q "uname -a;date;exit" >> abc.txt [ssuri@uslinuxnix01:~] $ cat abc.txt Linux uslinuxnix05.example.org 2.6.32-696.6.3.el6.x86_64 #1 SMP Fri Jun 30 13:24:18 EDT 2017 x86_64 x86_64 x86_64 GNU/Linux Tue Feb 20 06:59:14 UTC 2018 Linux uslinuxnix02.dev.example.org 3.10.0-693.2.2.el7.x86_64 #1 SMP Sat Sep 9 03:55:24 EDT 2017 x86_64 x86_64 x86_64 GNU/Linux Tue Feb 20 06:59:14 UTC 2018
We included the date command in the above example to further illustrate that both commands get executed over separate SSH connections at the same time in parallel.
This concludes our demonstration of simulating parallel process execution in Linux by putting processes in the background by appending the ampersand character after the command. We hope that you’ve found this article to be useful and we look forward towards your feedback.
Latest posts by Sahil Suri (see all)
- Enabling passwordless authentication for chrooted sftp users in Linux - August 15, 2018
- GitHub and git integration: using ssh instead of https - August 14, 2018
- How to use bash aliases over ssh in Linux - August 9, 2018
- Troubleshooting chrooted sftp account setup in Linux - August 8, 2018
- Using ShellCheck to write better shell scripts - August 6, 2018