Thuta Learning
IntermediateDevOps & Toolsbeginner

Artifacts and Passing Data Between Jobs

What you'll walk away with

  • Explain the core ideas behind Artifacts and Passing Data Between Jobs
  • Read the diagram and trace how the pipeline flows and what gates each stage
  • Read the workflow file and predict which jobs run, in what order, and what gates them

Build the mental model

Each job in a workflow gets its own runner. When the build job finishes, that machine is destroyed: its filesystem, its environment variables, its node_modules. A deploy job declared with needs: build inherits nothing except the ordering. This catches people out, because within a single job the steps share a working directory and the workflow feels like one continuous machine. It is not. It is a set of independent machines with a dependency graph drawn between them.

GitHub gives you two ways to move data across that boundary, and they are sized for different things. Artifacts are files. upload-artifact@v4 zips a path and stores it against the workflow run; download-artifact@v4 in a later job pulls it back onto the new runner. This is how compiled output, test reports, coverage files, and failure screenshots travel. Job outputs are strings. A job declares outputs: at job level, wires each one to a step that wrote key=value into the $GITHUB_OUTPUT file, and a downstream job reads it as needs.<job>.outputs.<name>. Version numbers, image tags, and computed flags belong here.

Choosing wrongly is the common failure. Uploading a one-line file as an artifact to move a version string works, but it adds a zip round-trip to every run for no reason. Trying to move a directory through job outputs does not work at all: outputs are strings, they are size-capped, and they are visible in logs, which also makes them the wrong place for anything sensitive.

Artifacts cost storage against your account, billed by size and duration, which is why retention-days exists. The default retention is generous, and a nightly job uploading a large bundle every night will quietly build a bill nobody looks at.

text
ARTIFACT HANDOFF BETWEEN JOBS
-----------------------------
  JOB build (runner A)              JOB deploy (runner B)
  +------------------------+        +------------------------+
  | npm ci                 |        | download-artifact      |
  | npm run build -> dist/ |        |   name: dist -> dist/  |
  | upload-artifact        |        | ./deploy.sh dist       |
  |   name: dist           |        +------------------------+
  +------------------------+                    ^
              |                                 |
              v                                 |
  +--------------------------------------------------------+
  | artifact store    name: dist    retention-days: 7       |
  +--------------------------------------------------------+

  job outputs (small strings, no storage cost):
    build.outputs.version ---> needs.build.outputs.version

  runner A is DESTROYED before runner B starts.
  Nothing on disk carries over; only artifacts and outputs do.

Connect it to a real scenario

A typical shape is one build job that compiles and one deploy job that ships exactly what was compiled. The important property is that deploy never rebuilds. If it did, you would be deploying a second, separately produced artifact, and any nondeterminism in your build, whether a timestamp, a lockfile resolved a minute later, or a different runner image, makes the thing you tested and the thing you shipped two different bundles.

So build runs npm run build, uploads dist/ as an artifact named dist, and separately computes the package version into $GITHUB_OUTPUT. The deploy job declares needs: build, downloads the dist artifact into ./dist, and passes needs.build.outputs.version to the deploy script.

Two details are worth internalising. Writing an output uses the $GITHUB_OUTPUT file, not the old set-output workflow command, which was disabled for security reasons and will simply do nothing if you copy it from an old blog post. And upload-artifact@v4 will not append to an existing artifact name the way v3 did, so uploading twice under the same name within one run is an error. Give matrix jobs distinct artifact names that include the matrix values, and merge them in a later job if you genuinely need a single bundle.

Try the working example

yaml
name: Build then deploy

on:
  push:
    branches: [main]

jobs:
  build:
    runs-on: ubuntu-latest
    outputs:
      version: ${{ steps.meta.outputs.version }}
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: npm

      - name: Install dependencies
        run: npm ci

      - name: Build
        run: npm run build

      - name: Record the package version as a job output
        id: meta
        run: |
          VERSION=$(node -p "require('./package.json').version")
          echo "version=$VERSION" >> "$GITHUB_OUTPUT"

      - name: Upload the build output
        uses: actions/upload-artifact@v4
        with:
          name: dist
          path: dist/
          retention-days: 7

  deploy:
    needs: build
    runs-on: ubuntu-latest
    steps:
      - name: Download the build output
        uses: actions/download-artifact@v4
        with:
          name: dist
          path: dist

      - name: Deploy exactly what was built
        run: ./scripts/deploy.sh dist "${{ needs.build.outputs.version }}"
You should see
Two jobs run in sequence, not in parallel: deploy declares needs: build, so it does not start until build has finished successfully. build checks out the code, installs, compiles into dist/, records the package version as a job output, and uploads dist/ to the run's artifact store with a seven-day retention. If build fails, deploy is skipped entirely and no deployment happens. When build succeeds, deploy starts on a completely new runner with an empty filesystem, and it does not check out the repository at all, so the only reason dist/ exists there is the download-artifact step. The version string reaches the deploy script through needs.build.outputs.version rather than through any file on disk. The uploaded artifact is also downloadable from the run's summary page for seven days, which is what makes this pattern equally useful for test reports and for deployments.

5-minute try-it

Split an existing single-job workflow into build and deploy. First run deploy without the download step and confirm that dist/ genuinely does not exist on the second runner. Then add the artifact upload and download and get it working. Finally, move a value you currently write into a file, such as a version or a commit short SHA, into a job output instead, and set retention-days to the shortest value your team actually needs.

One important caution

Assuming a later job can read files an earlier job wrote: each job is a new runner, so without upload-artifact and download-artifact the directory simply is not there.

Uploading under the same artifact name from several matrix jobs: upload-artifact@v4 does not merge same-named uploads the way v3 did, so the run fails and the names need the matrix values in them.

GitHub Docs - Storing and sharing data from a workflowCI/CD with GitHub Actions

Easy traps

  • Assuming a later job can read files an earlier job wrote: each job is a new runner, so without upload-artifact and download-artifact the directory simply is not there.
  • Uploading under the same artifact name from several matrix jobs: upload-artifact@v4 does not merge same-named uploads the way v3 did, so the run fails and the names need the matrix values in them.
  • Validate a workflow on a branch or test repository before pointing it at a production deployment.

Exercise

Split an existing single-job workflow into build and deploy. First run deploy without the download step and confirm that dist/ genuinely does not exist on the second runner. Then add the artifact upload and download and get it working. Finally, move a value you currently write into a file, such as a version or a commit short SHA, into a job output instead, and set retention-days to the shortest value your team actually needs.

You'll know it worked when: Two jobs run in sequence, not in parallel: deploy declares needs: build, so it does not start until build has finished successfully. build checks out the code, installs, compiles into dist/, records the package version as a job output, and uploads dist/ to the run's artifact store with a seven-day retention. If build fails, deploy is skipped entirely and no deployment happens. When build succeeds, deploy starts on a completely new runner with an empty filesystem, and it does not check out the repository at all, so the only reason dist/ exists there is the download-artifact step. The version string reaches the deploy script through needs.build.outputs.version rather than through any file on disk. The uploaded artifact is also downloadable from the run's summary page for seven days, which is what makes this pattern equally useful for test reports and for deployments.

Artifacts and Passing Data Between Jobs | Thuta Learning