Pipes, Redirection, and Here-Documents

Build the mental model

Pipes and redirection are how bash connects independent programs together instead of forcing you to write everything as one monolithic script — this is the Unix philosophy in practice: small, focused tools, composed. A pipe (|) connects one command's stdout directly to the next command's stdin, so printf ... | wc -l counts lines without ever writing an intermediate file, and grep -v banana | sort chains a filter into a sort with no temporary state in between — each command in the chain runs concurrently, reading and writing through the pipe as data flows. Redirection operators work on files instead of other commands: > sends a command's stdout to a file, overwriting whatever was there, while >> appends to the end instead, preserving existing content — mixing these up is a common way to accidentally destroy a log file's history. The reason stdout and stderr can be redirected independently, as in 1>out.txt 2>err.txt, is that they are genuinely separate streams under the hood, identified by file descriptors 1 and 2 respectively; 2>&1 merges stderr into wherever stdout is currently pointed, and redirecting unwanted output to /dev/null discards it entirely while still letting the rest of the script's output through normally. Here-documents (<<EOF ... EOF) and here-strings (<<<) invert the whole idea: instead of a command's output going somewhere, they feed literal text into a command's stdin, which is how you generate multi-line content or supply a single value without needing a separate file on disk.

Connect it to a real scenario

A typical CI or deployment script might pipe a command's output through grep to check for an error keyword, redirect noisy-but-unimportant output to /dev/null while still capturing errors for later, or use a here-document to generate a file inline — this lesson's code demonstrates each of these individually so you can see the mechanics clearly. printf "apple\nbanana\ncherry\n" | wc -l shows the simplest pipe: three lines in, a count of 3 out, no file involved. grep -v banana | sort chains further, removing one line and then sorting what remains, which is how a small pipeline replaces what would otherwise be several lines of manual filtering and looping. The redirection block deliberately separates stdout and stderr into two different files using 1>"$out_file" 2>"$out_file.err" inside a { ... } group, which is the pattern you'd use in a real script to keep normal output and error output in separate logs — and then shows the opposite, 2>&1, merging both into the same file so both messages land together, which is what you'd want when debugging and don't want to check two files. The /dev/null example shows how to silence one specific line's output without affecting the rest of the script's normal output around it. Finally, the here-document feeds a fixed three-line message into cat's stdin without a separate template file, and the here-string reads a single value, "hello-there", directly into the word variable via read -r — a quick way to feed one value into a command that expects to read from stdin rather than take an argument.

• | pipes one command's stdout into the next command's stdin, building a processing chain
• > overwrites and >> appends to a file; 2>&1 merges stderr into stdout, and &> or > file 2>&1 sends both to the same place
• <<EOF ... EOF feeds multi-line text to a command's stdin (a here-document); <<< feeds a single value (a here-string)

Try the working example

#!/bin/bash
echo "Piping ls-like data through wc:"
printf "apple\nbanana\ncherry\n" | wc -l

echo ""
echo "Chaining grep and sort in a pipeline:"
printf "banana\napple\ncherry\n" | grep -v banana | sort

echo ""
echo "Redirecting stdout to a file, then reading it back:"
out_file=$(mktemp)
echo "saved output" > "$out_file"
cat "$out_file"

echo ""
echo "Appending more output to the same file:"
echo "second line" >> "$out_file"
cat "$out_file"

echo ""
echo "Redirecting stderr separately from stdout:"
{ echo "this is stdout"; echo "this is stderr" >&2; } 1>"$out_file" 2>"$out_file.err"
echo "stdout file contains:"
cat "$out_file"
echo "stderr file contains:"
cat "$out_file.err"

echo ""
echo "Discarding output with /dev/null:"
echo "you will not see this" > /dev/null
echo "but this line still prints"

echo ""
echo "Combining stdout and stderr with 2>&1:"
{ echo "normal message"; echo "error message" >&2; } > "$out_file" 2>&1
cat "$out_file"

echo ""
echo "Here-document: sending multi-line input to cat"
cat <<EOF
Line one
Line two
Line three
EOF

echo ""
echo "Here-string: sending a single value to a command"
read -r word <<< "hello-there"
echo "Read from here-string: $word"

rm -f "$out_file" "$out_file.err"

Running this script prints:

Piping ls-like data through wc:
3

Chaining grep and sort in a pipeline:
apple
cherry

Redirecting stdout to a file, then reading it back:
saved output

Appending more output to the same file:
saved output
second line

Redirecting stderr separately from stdout:
stdout file contains:
this is stdout
stderr file contains:
this is stderr

Discarding output with /dev/null:
but this line still prints

Combining stdout and stderr with 2>&1:
normal message
error message

Here-document: sending multi-line input to cat
Line one
Line two
Line three

Here-string: sending a single value to a command
Read from here-string: hello-there

GNU Bash Manual: Redirections — Bash / Shell Scripting