Debugging Scripts with set -x and ShellCheck

Build the mental model

When a script misbehaves and it's not obvious why, set -x turns on execution tracing: bash prints every command it's about to run to stderr (so it won't corrupt anything the script writes to stdout), prefixed by the value of PS4 (+ by default), and crucially, with all variables, globs, and substitutions already expanded to their real values. That distinction matters: a trace line showing cp file1.txt file2.txt tells you exactly what ran, not just what the source code said to run, which is invaluable when the bug is that a variable expanded to something unexpected — an empty string, a value with a stray space, a glob that matched more files than intended. You can scope tracing narrowly by wrapping only the suspicious section between set -x and set +x rather than tracing an entire script and drowning in output. Customizing PS4 — for example PS4='+ [line ${LINENO}] ' — adds the source line number to every trace line, turning a long, otherwise anonymous wall of + commands into something you can map straight back to the file; PS4 can also reference ${FUNCNAME[0]} or ${BASH_SOURCE[0]} for scripts with multiple files or deep call stacks. ShellCheck takes a complementary, static approach: instead of watching a script run, it parses the script's text and flags likely bugs before you ever execute a single line — unquoted variables that break on spaces or glob unexpectedly, comparing strings with the wrong operator, using = when you meant ==, reading command output into a for loop instead of while read, and dozens of other well-documented shell footguns, each with a numbered SC code you can look up for a full explanation.

Connect it to a real scenario

You're maintaining a deploy script that works fine on your machine but does the wrong thing silently in CI, and there's no obvious error message pointing at why. Rather than sprinkling echo statements through the script and re-running the whole CI pipeline after every guess, running bash -x deploy.sh in CI produces a full trace of every command as it actually executed there, with real values substituted in — showing you exactly which branch of an if-statement was taken and what each variable expanded to at that exact point in that exact environment. That's often the whole diagnosis: an environment variable CI sets differently than your local shell does turns out to be the real culprit, something no amount of staring at the source code locally would reveal, because locally that variable never had the problematic value. Running the same script through shellcheck deploy.sh before it ever reaches CI can catch the underlying mistake even earlier — for instance flagging an unquoted $CI_BRANCH that would silently break the moment a branch name contains a slash or a space, long before that particular branch name ever gets created and triggers the bug in practice.

• set -x traces commands with expanded values as they run; set +x turns it back off
• PS4 customizes the trace prefix — adding ${LINENO} makes long traces easy to map back to source
• ShellCheck (shellcheck.net) statically finds quoting bugs and other mistakes before you run the script

Try the working example

#!/bin/bash
set -euo pipefail

add() {
  local a=$1
  local b=$2
  echo $((a + b))
}

echo "before tracing"

PS4='+ [line ${LINENO}] '
set -x
result=$(add 2 3)
set +x

echo "after tracing, result was: $result"

Running the script above prints:

before tracing
++ [line 14] add 2 3
++ [line 5] local a=2
++ [line 6] local b=3
++ [line 7] echo 5
+ [line 14] result=5
+ [line 15] set +x
after tracing, result was: 5

ShellCheck — a shell script static analysis tool — Bash / Shell Scripting