In this lesson, you will learn how to consolidate the output of multiple running processes into a single log file using a powerful technique called process substitution.
The Challenge of Multiple Streams
When you have several programs running simultaneously, each naturally directs its output to its own file descriptor (usually standard output or standard error). If you want to monitor these outputs in one place, you might be tempted to create multiple log files and "tail" them, but that gets messy quickly. We need a way to make the shell treat the output of several processes as if they were a single input stream.
Using Process Substitution
Process substitution is a shell feature that allows the output of a command to appear as a file to other commands. You invoke it using the syntax <(command). The shell executes the command in the background and provides a temporary file path (like /dev/fd/63) that represents the output stream of that command.
You can combine multiple streams into one by using the cat command with multiple process substitutions:
# Consolidating logs from two separate processes into one file
cat <(process_one) <(process_two) > consolidated.log
In this example, cat receives two file paths created by the shell. It reads from both simultaneously, interleaving their data as it arrives, and redirects the combined result into consolidated.log.
Why This Works
Because the shell treats these substitutions as standard file descriptors, you can use them with any tool that accepts file inputs. This strategy effectively "multiplexes" the data streams without you having to manually manage temporary intermediate files or complex pipe structures. The shell handles the cleanup of these temporary descriptors automatically once the commands complete. This keeps your process environment clean and avoids the pitfalls of managing scattered, per-process log files.
Key Takeaway: Process substitution allows you to turn the output of multiple concurrent commands into readable file streams, enabling you to merge disparate data sources into a single unified output stream with ease.