Learn BASH Scripting - Command Substitution, Process Substitution & HereDoc

#Introduction

In episode 16 we covered String Manipulation & Parameter Expansion — how to measure, slice, strip, and replace text right inside the shell — so you can now process data that's already in variables. The big next question is: where does that data come from? A script that only works with manually typed data will never be useful for automation. A useful script takes data from the output of other commands, from file contents, or from interaction between processes.

This episode 17 answers those three core needs. First, command substitution $(...): a way to capture a command's output and turn it into a variable's value — the heart of every script that "takes something from the outside world". Second, process substitution <(cmd) and >(cmd): a way to treat command output as if it were a file, without ever creating a temporary file on disk. Third, here-documents and here-strings: a way to write long text (configuration files, SQL, HTML) directly inside the script, neatly.

Imagine you're a chef preparing a restaurant. Command substitution is the way you ask another kitchen station to prep ingredients and receive them straight in a container; process substitution is the way you have two stations prepare two lists and then compare them on the table without writing on a board first; a here-document is the way you write a complete recipe on a single card so it isn't scattered across many pieces of paper. All three make your scripts far more expressive — and more importantly, they remove the need for temporary files that are prone to clutter.

Let's start from the most fundamental: how a command can "return a value" into your script.

#Main Discussion

#Command Substitution: Capturing Output as a Value

In BASH, almost every command prints output to stdout. Command substitution is the mechanism to capture that output and place it wherever you write $(...). The simplest example:

tanggal=$(date +%Y-%m-%d)
echo "Hari ini: $tanggal"
files=$(ls)
echo "Jumlah file: $(wc -l <<< "$files")"

Hari ini: 2026-08-02
Jumlah file: 14

The $(command) form is the modern and recommended way. There's also a legacy form from the Bourne Shell era: backticks `command`. Backticks still work in modern BASH, but they are deprecated and should never be used in new scripts. Why?

• Parentheses are easier to read and nest. A backtick inside a backtick has to be escaped with \``, which is dreadful: `` ls `pwd```. With parentheses, nesting is written plainly:$(ls $(pwd))`.
• Parentheses use normal quoting rules. Inside $(...), quotes work exactly as you'd expect. Inside backticks, backslashes and quotes often behave unexpectedly.
• ShellCheck and most style guides (Google Shell Style Guide, ShellCheck) explicitly forbid backticks.

So the rule is simple: output=$(command) — period. Backticks are only worth knowing so you can read old scripts without misreading them.

Command substitution can also be nested as deep as needed, and expansions inside it are evaluated layer by layer from the inside out — just like evaluating parentheses in math:

host_ip=$(hostname -I | awk '{print $1}')
echo "IP pertama server: $host_ip"
repo_dir="$(dirname "$(which bash)")"
echo "Direktori bash: $repo_dir"

Note one important behavior: command substitution strips all trailing newlines from the output. If a command prints "hasil\n\n", the variable will contain "hasil" — trailing newlines are discarded. That's usually what you want, but it can be confusing when the command output really does end with a meaningful blank line.

One more difference that often confuses people — and makes the pitfalls list at the end of the episode: $(...) is not ${...}. $() runs a command; ${} expands a variable or parameter expansion. Writing ${date} when you mean $(date) yields the value of the date variable (which is empty), not the date. These two symbol pairs are "conjoined twins" you must always consciously tell apart.

#Process Substitution: Output as a "File"

Sometimes a tool refuses to take input through a pipe — it demands a file path as an argument. A classic example: diff compares two files, comm needs two sorted files. Manually, you'd write output to temporary files, compare, then delete them — three steps, and easy to forget to clean up.

Process substitution eliminates all of that. Its forms are:

• <(command) — the output of command is presented as if it were a readable file. BASH creates a /dev/fd/N file descriptor behind the scenes.
• >(command) — the reverse: it presents a writable file; whatever you write to it is forwarded as input to command.

The most famous example — comparing the contents of two directories without temporary files:

diff <(ls /var/www/site-a) <(ls /var/www/site-b)

2c2
< index.old.html
---
> index.html
5d4
< backup.sql.gz

That line reads: "in directory A there are index.old.html and backup.sql.gz that aren't in B; in B there's index.html that isn't in A". Without process substitution, you'd have to write two temporary files, run diff, then clean up. With <(cmd), it's all one line — and there's no risk of leftover temporary files.

Process substitution is also very useful for comm (the command for comparing two sorted data sets) and for while read patterns that need to read two streams at once:

comm -3 <(awk '{print $1}' file-A.txt | sort -u) <(awk '{print $1}' file-B.txt | sort -u)

This prints lines that exist in only one of the files — without ever creating a merged file. In the real world, this pattern is used to detect differences between IP lists, user lists, or domain lists from two data sources.

#Here-Documents: Writing Long Text Cleanly

Now on to the third weapon: the here-document (often shortened to heredoc). Its syntax resembles serving text inside a script:

cat << EOF
Selamat datang di server $(hostname).
Kapasitas disk saat ini:
$(df -h / | tail -1)
EOF

Selamat datang di server webserver-01.
Kapasitas disk saat ini:
/dev/mapper/root  100G  42G   58G  43% /

The << EOF pattern reads: "start writing text, continue until you find a line whose content is exactly EOF". Everything between the delimiters is streamed to the stdin of the command in front of it (cat). The text can be dozens of lines long — this replaces writing many echo commands, which is impractical and prone to quoting mistakes.

Note in the example above: $(hostname) and $(df -h / | tail -1) are expanded — because the EOF delimiter is written without quotes. This is a feature, not a bug: an unquoted heredoc is a live template. But this is exactly where the hidden trap lies — you don't always want expansion to happen.

To write text that must be literally verbatim — config file contents with $, backticks, or template syntax — quote the delimiter:

cat << 'EOF'
Ini literal: $HOME, $(date), dan `pwd` tidak akan diekspansi.
Cocok untuk menulis template, konfigurasi, atau kode program.
EOF

Ini literal: $HOME, $(date), dan `pwd` tidak akan diekspansi.
Cocok untuk menulis template, konfigurasi, atau kode program.

The rule of thumb: unquoted = expansion; quoted = literal. And there's a third variant for indentation cleanliness: <<- EOF lets the text lines start with a tab (not spaces) that is automatically stripped — useful when a heredoc sits inside an already-indented function, keeping the code tidy without ruining the text format.

One more form that's similar but different: the here-string <<<. It sends a string (not a text block) to stdin, and automatically appends a trailing newline. It's the most concise way to feed input to commands that only read from stdin:

grep -o 'GET[^"]*' <<< "$(cat access.log)"
md5sum <<< "pesan rahasia"

Here-strings are very useful for md5sum, sha256sum, grep, and other commands that can't accept input via positional arguments. The $(wc -l <<< "$files") pattern we used at the start of the episode also takes advantage of it — wc reads from stdin, and the here-string injects the $files string without writing a file.

#Real Practice: Generating Config Files from Inside a Script

Now let's combine everything in one production scenario: a deployment script that must create an Nginx configuration file for a new site, with dynamic values (domain name, root directory, upstream port). Notice how the unquoted heredoc becomes a template and the quoted heredoc keeps the parts that must be literal:

#!/bin/bash
 
DOMAIN="${1:?Domain wajib diberikan, mis. ./deploy-site.sh example.com}"
ROOT_DIR="/var/www/${DOMAIN}"
UPSTREAM_PORT=3000
 
mkdir -p "$ROOT_DIR"
 
cat > "/etc/nginx/sites-available/${DOMAIN}" << EOF
server {
    listen 80;
    server_name ${DOMAIN} www.${DOMAIN};
 
    root ${ROOT_DIR};
    index index.html;
 
    location / {
        proxy_pass http://127.0.0.1:${UPSTREAM_PORT};
        include /etc/nginx/proxy_params;
    }
}
EOF
 
echo "Konfigurasi untuk ${DOMAIN} berhasil dibuat di /etc/nginx/sites-available/"

Let's dissect the important things in the script above:

• cat > path << EOF — the heredoc output is written to a file (not stdout) because of the > redirection.
• ${DOMAIN}, ${ROOT_DIR}, ${UPSTREAM_PORT} are expanded because the delimiter isn't quoted — this is what lets one template serve many sites.
• ${1:?message} (from episode 16) ensures the user must provide a domain argument, otherwise the script stops.
• The proxy_pass part and the server block are written in the template — all literal except the variables we inject.

If you have a part that must be truly literal — for example writing a script inside a script, or a template containing $ belonging to another program — use a quoted delimiter for that part.

#Classic Pitfalls

1. Forgetting to write the closing delimiter. BASH reads a heredoc until it finds a line that is exactly the delimiter. If you forget to write EOF at the end, BASH keeps "waiting" until the end of the file — and you get an unexpected end of file error. Solution: get into the habit of writing the closer immediately after the opener, then filling in the middle.

2. Trailing whitespace on the closing line. A line EOF with one trailing space (or space indentation on <<-, which requires tabs) won't be recognized as the closer. This is the most common cause of a "stuck" heredoc. Check with cat -A if necessary to see hidden characters.

3. Unquoted delimiter → unexpected expansion. This is the favorite bug source: you write a config template containing $ (for example another application's environment variables, or $ANSI color syntax), and the heredoc innocently expands it to empty. If you mean literal, quote the delimiter (<< 'EOF'). If you mean template, quote the variables you want expanded inside the heredoc.

4. Mixing up $(...) and ${...}. $(cmd) runs a command; ${var} reads a variable. The two are often swapped by script authors seeing them for the first time. When a variable's output is mysteriously empty, ask first: did you write $() when you meant ${}?

5. Backticks that slip past review. Old scripts containing `command` still run — so the bug isn't visible. But when nested or in tricky quoting, backticks become a source of bizarre behavior. If you see backticks in a script you're refactoring, replace them with $(...) — it's one of the safest fixes you can make.

#Conclusion

In this episode 17 you've learned three ways scripts take in and put out data from the real world: command substitution $(...) to capture command output into a variable (and why backticks are obsolete), process substitution <(cmd) and >(cmd) to treat output as a file without touching the disk, plus here-documents << EOF and here-strings <<< for writing long text and feeding multiline input cleanly.

The key takeaways:

• output=$(command) is the modern standard; avoid backticks.
• Expansion happens in a heredoc only if the delimiter isn't quoted; quote the delimiter for literal text.
• Process substitution presents output as a file descriptor — fewer steps, zero temporary files.
• Watch for trailing whitespace on the closing delimiter and always distinguish $() from ${}.
• Command substitution runs in a subshell — avoid mass calls inside tight loops.

Now you can take data from other commands and print clean artifacts. But there's still one chasm to bridge: what happens when something fails? A script that runs smoothly on a laptop suddenly breaks everything on a server — because its commands fail in ways we never anticipated. In episode 18 we'll build resilience with Error Handling & Robustness (Unofficial BASH Strict Mode): understanding exit codes, set -euo pipefail, allowing intentionally acceptable failures, and reinforcing your scripts layer by layer until they're hard to break silently. See you there!