202511241417bash-process-substitution

Command Substitution（命令替换）

Command substitution

Command substitution allows us to use the output of a command as an expansion:

[me@linuxbox ~]$ echo $(ls)
Desktop Documents ls-output.txt Music Pictures Public Templates Videos
 
[me@linuxbox ~]$ ls -l $(which cp)
-rwxr-xr-x 1 root root 71516 2012-12-05 08:58 /bin/cp

There is an alternative syntax for command substitution in older shell programs that is also supported in bash . It uses back quotes instead of the dollar sign and parentheses:

[me@linuxbox ~]$ ls -l `which cp`
-rwxr-xr-x 1 root root 71516 2012-12-05 08:58 /bin/cp

指向原始笔记的链接

Subshell（子shell）

Definition: A subshell is a child process launched by a shell (or shell script).

一个最基本的motivation是，当你在shell脚本中写一些函数，这些函数如果涉及到切换目录，那函数执行完之后，当前目录到底有没有切换？

结论：如果使用子shell，切换目录的事情都在子shell发生，不会影响当前shell。如果没有用子shell，则当前shell的目录会受影响。

Note

这里我们说的shell其实就是一个bash/sh进程。进程有父子，bash进程（A）启动（fork）了另一个bash进程（B），那么B就是A的子shell，A就是B的父shell。

子shell一般用小括号()包裹，另一个常见的形式是管道符|，管道符之后的一般是子shell。

demo() {
	echo $(pwd) # /a/b/c
	( # launch a subshell to run the following
		cd haha
		echo $(pwd) # /a/b/c/haha
	)
	echo $(pwd) # /a/b/c
}
 
foo() {
	cd haha
}
 
foo_sub() ( # note here we use '('
	cd haha
)
 
bar() {
	echo $(pwd) # /a/b/c
	foo
	echo $(pwd) # /a/b/c/haha
}
 
bar1() {
	echo $(pwd) # /a/b/c
	foo_sub     # this will launch a subshell
	echo $(pwd) # /a/b/c, so pwd here will not change
}

再来看一个陷进，

#!/bin/bash
# Pitfalls of variables in a subshell.
 
outer_variable=outer
echo
echo "outer_variable = $outer_variable"
echo
 
(
# Begin subshell
 
echo "outer_variable inside subshell = $outer_variable"
inner_variable=inner  # Set
echo "inner_variable inside subshell = $inner_variable"
outer_variable=inner  # Will value change globally?
echo "outer_variable inside subshell = $outer_variable"
 
# Will 'exporting' make a difference?
#    export inner_variable
#    export outer_variable
# Try it and see.
 
# End subshell
)
 
echo
echo "inner_variable outside subshell = $inner_variable"  # Unset.
echo "outer_variable outside subshell = $outer_variable"  # Unchanged.
echo
 
exit 0
 
# What happens if you uncomment lines 19 and 20?
# Does it make a difference?
 
# Answer by yychi: No! 'export' in subshell does not transfer changes to parent shell.

In general, an external command in a script forks off a subprocess, whereas a Bash builtin does not. For this reason, builtins execute more quickly and use fewer system resources than their external command equivalents.

参考：https://tldp.org/LDP/abs/html/subshells.html

Tip

如无必要，不要使用子shell，因为有性能损失。组命令{ command1; command2; [command3; ...] }运行更快且占用内存更低。另，花括号和命令之间的空格是必须的。

Process Substitution（进程替换）

将一个命令的输出作为另一个命令的输入是一种很常见的手段。

# 文件排序、去重
cat a.txt | sort | uniq
 
# 对所有csv文件计算md5
ls *.csv | xargs -i md5sum {}
 
# 找出所有路径包含somedir的abc.cfg文件
find -name "*abc.cfg*" | grep somedir

想一想，如果要将多个命令的输出，作为一个命令的输入，要怎么做呢？

这种情况就得用进程替换了！

进程替换的语法如下，

>(command_list)
<(commamd_list)

注意小于/大于号和括号之间不能有空格，否则语法错误。

进程替换的实质是用

• /dev/fd/<n>（旧版）
• /proc/self/fd/<n>（新版）

文件描述符作为临时文件，将括号里面命令的输出发送到另一个进程。

$ echo >(true)
/proc/self/fd/13
$ echo <(true)
/proc/self/fd/11
 
$ wc <(cat /usr/share/dict/words)
 102401  102401  972398 /proc/self/fd/11
 
$ grep script /usr/share/dict/words | wc
     67      67     845
$ wc <(grep script /usr/share/dict/words)
     67      67     845 /proc/self/fd/11

来看一个经典的Subshell（子shell）陷进，

echo haha | read reply
echo $reply # 输出为空

这是因为管道命令一般在子shell执行，等于管道之后的命令会fork当前的shell环境，执行，返回。但子shell里面做的修改是无法传递到当前shell进程的。所以，当前shell压根没有reply变量。

使用进程替换可以规避，

$ read reply < <(echo haha)
#            ^ ^ First < is redirection, second is process substitution.
$ echo $reply
haha

再来看一个更复杂的例子，

#!/bin/bash
# wr-ps.bash: while-read loop with process substitution.
 
# This example contributed by Tomas Pospisek.
# (Heavily edited by the ABS Guide author.)
 
echo
 
echo "random input" | while read i
do
  global=3D": Not available outside the loop."
  # ... because it runs in a subshell.
done
 
echo "\$global (from outside the subprocess) = $global"
# $global (from outside the subprocess) =
 
echo; echo "--"; echo
 
while read i
do
  echo $i
  global=3D": Available outside the loop."
  # ... because it does NOT run in a subshell.
done < <( echo "random input" )
#    ^ ^
 
echo "\$global (using process substitution) = $global"
# Random input
# $global (using process substitution) = 3D: Available outside the loop.
 
 
echo; echo "##########"; echo
 
 
 
# And likewise . . .
 
declare -a inloop
index=0
cat $0 | while read line
do
  inloop[$index]="$line"
  ((index++))
  # It runs in a subshell, so ...
done
echo "OUTPUT = "
echo ${inloop[*]}           # ... nothing echoes.
 
 
echo; echo "--"; echo
 
 
declare -a outloop
index=0
while read line
do
  outloop[$index]="$line"
  ((index++))
  # It does NOT run in a subshell, so ...
done < <( cat $0 )
echo "OUTPUT = "
echo ${outloop[*]}          # ... the entire script echoes.
 
exit $?

参考：https://tldp.org/LDP/abs/html/process-sub.html

---

see also: Brief Introduction to Shell Script - comments