On Mon, Jan 06, 2020 at 01:30:40PM +1100, Rodney Brown wrote:

https://www.spinellis.gr/blog/20191214/ Convert file I/O into pipe I/O with /dev/fd

Some Unix commands read data from files or write data to files, without offering an obvious way to use them as part of a pipeline. How can you write a program to interact with such a command in a streaming fashion?

This would allow your program and the command run concurrently, without the storage and I/O overhead of a temporary file. You could create and use a named pipe, but this is a clunky solution, requiring you to create and destroy a unique underlying file name. Here's a better approach.

Modern Unix systems offer a virtual device directory named /dev/fd/. The files in it correspond to the file descriptors of each process. ...

This is known as Process Substitution (because it substitutes a filename arg with a process - which can be any command or list or pipeline of commands that either output to stdout or consume stdin). Several shells have built-in support for it, including bash, some versions of ksh, and zsh. Note that Process Substitution is not available when bash is run in POSIX mode - e.g. when /bin/sh is a symlink to bash and your script uses #!/bin/sh instead of #!/bin/bash, or when POSIXLY_CORRECT is set, or if you use 'set -o posix', etc. See also: https://en.wikipedia.org/wiki/Process_substitution http://www.tldp.org/LDP/abs/html/process-sub.html https://mywiki.wooledge.org/ProcessSubstitution The wikipedia link above has a very nice, short and clear explanation of what it is, why it's useful, and how to use it. There are also over 100 questions (with answers) about Process Substitution on U&L with lots of good examples and explanations: https://unix.stackexchange.com/questions/tagged/process-substitution This Q is a pretty good one: https://unix.stackexchange.com/questions/17107/process-substitution-and-pipe Finally, it's documented in the bash man page: Process Substitution Process substitution allows a process's input or output to be referred to using a filename. It takes the form of <(list) or >(list). The process list is run asynchronously, and its input or output appears as a filename. This filename is passed as an argument to the current command as the result of the expansion. If the >(list) form is used, writing to the file will provide input for list. If the <(list) form is used, the file passed as an argument should be read to obtain the output of list. Process substitution is supported on systems that support named pipes (FIFOs) or the /dev/fd method of naming open files. When available, process substitution is performed simultaneously with parameter and variable expansion, command substitution, and arithmetic expansion. craig -- craig sanders <cas@taz.net.au>