Learn GitLab CI/CD - Accelerating Build Time with Caching

#Introduction

In episode 7 we covered artifacts — how to store build outputs and transfer them between jobs. Many people assume artifacts also speed up pipelines, but they serve a different purpose. The reality: your pipeline is still slow every time it runs npm install or mvn package, because jobs download and reinstall all dependencies from scratch even when nothing changed. One small code change can mean minutes wasted just waiting for node_modules to be rebuilt.

This episode covers cache — GitLab's mechanism for storing temporary dependencies so the next job doesn't have to rebuild from scratch. You'll learn when to use artifacts and when to use cache, how to configure caching correctly, and how to set up distributed caching with object storage so the cache stays useful even when your jobs run on different runners.

#Main Discussion

#Artifacts vs Cache: Two Different Things

The biggest confusion among GitLab CI beginners is thinking artifacts and cache are the same. In reality they have opposite purposes:

• Artifacts store the final results of a job — binaries, zips, reports — used if needed by subsequent jobs or downloaded by humans. Think of it like a delivered package after the order is done.
• Cache stores temporary dependencies — node_modules, ~/.m2, pip cache — that jobs always need to work fast. Think of it like a raw-material warehouse filled once and reused by many processes.

The rule for choosing is simple: you need the result to process or download → artifacts. You need to speed up dependency installation → cache. The two are often used together — for example a build job caches node_modules (so npm ci is fast) while producing dist/ as artifacts.

#Basic Cache Configuration

build_app:
  stage: build
  image: node:20-alpine
  cache:
    key: npm-cache
    paths:
      - node_modules/
  script:
    - npm ci
    - npm run build
  artifacts:
    paths:
      - dist/

Here key: npm-cache gives the cache archive a unique name. All jobs with the same key share one cache archive. paths determines which directory is cached — in this case node_modules/. When build_app finishes, node_modules/ is uploaded to the cache; when another job with the npm-cache key runs, its contents are downloaded before the script executes.

#Dynamic Cache Keys Based on Lock Files

The problem with a static key (npm-cache): if package-lock.json changes, the old cache containing outdated dependencies is still used — installation can fail because package versions don't match. The solution is a dynamic cache key that automatically changes when dependencies change:

build_app:
  stage: build
  image: node:20-alpine
  cache:
    key:
      files:
        - package-lock.json
    paths:
      - node_modules/
  script:
    - npm ci
    - npm run build

With key: files: [package-lock.json], GitLab computes a hash of the lock file's contents and uses it as the cache key. As long as package-lock.json doesn't change, the hash stays the same and the cache is used — as soon as the lock file changes (e.g. a new dependency), the hash changes and GitLab automatically creates a new cache. This guarantees cache matches without manually bumping key versions.

For Java/Maven projects, the pattern is identical — only the lock file differs:

test_maven:
  stage: test
  image: maven:3.9-eclipse-temurin-21
  variables:
    MAVEN_OPTS: "-Dmaven.repo.local=$CI_PROJECT_DIR/.m2/repository"
  cache:
    key:
      files:
        - pom.xml
    paths:
      - .m2/repository/
  script:
    - mvn verify

Notice MAVEN_OPTS moving the local Maven repository into the project workspace ($CI_PROJECT_DIR/.m2/repository) — this matters so paths: .m2/repository/ can capture it, because the default ~/.m2 sits outside the cacheable workspace. This is one of the classic Maven caching traps that makes the cache never get used.

#Policy: pull-push, pull, and push

The policy: keyword controls when the cache is downloaded and uploaded. Its three values:

• pull-push (default) — the cache is downloaded at the start of the job and uploaded again at the end. Most commonly used.
• pull — only downloads, doesn't upload. Good for jobs that use the cache but don't want to overwrite its contents (e.g. parallel test jobs that only read).
• push — only uploads, doesn't download. Good for jobs that build a fresh cache after dependencies change, while other jobs just pull.

test_app:
  stage: test
  image: node:20-alpine
  cache:
    key:
      files:
        - package-lock.json
    paths:
      - node_modules/
    policy: pull
  script:
    - npm test

The classic combination: one install job with policy: push that refreshes the cache when the lock file changes, and many test jobs with policy: pull that only consume the cache — so there's no race overwriting the cache between parallel jobs.

#Distributed Caching with Object Storage

By default, the cache is stored locally on the runner machine. Problems appear as soon as you have more than one runner: a cache created by runner A isn't available on runner B. For teams using many runners (especially with auto-scaling), the solution is a distributed cache — storing the cache in shared object storage (S3, MinIO, GCS) accessed by all runners.

The configuration happens in the runner machine's config.toml:

[[runners]]
  name = "docker-runner"
  executor = "docker"
  [runners.cache]
    Type = "s3"
    Path = "gitlab-cache"
    Shared = true
    [runners.cache.s3]
      ServerAddress = "s3.amazonaws.com"
      BucketName = "my-ci-cache"
      AccessKey = "AKIAXXXX"
      SecretKey = "S3CRET"

With Shared = true, all runners connected to the same bucket share the cache. ServerAddress can be swapped to an internal MinIO endpoint (http://minio:9000) if you're self-hosted — a common pattern at companies that don't want cache data leaving the public cloud. After this, the same key always produces a cache hit no matter which runner executes the job — parallel pipelines across runners become consistent and fast.

#Closing

In this episode 8, you've understood caching thoroughly:

• Artifacts store final results for processing/downloading, while cache stores temporary dependencies to speed up execution — they have different roles and are often used together.
• cache: key:, paths: defines a cache; key: files: creates a dynamic key from lock files (package-lock.json, pom.xml) so the cache automatically refreshes when dependencies change.
• policy: pull-push / pull / push controls when the cache is downloaded and uploaded, preventing races between parallel jobs.
• Distributed caching connects runners to S3/MinIO so all runners share the same cache — a must for teams with many runners.

With the right cache, npm ci and mvn verify that once took minutes can drop to seconds. But there's another factor slowing your pipeline without you realizing it: the linear stage structure. In episode 9 we'll cover Directed Acyclic Graph (DAG) pipelines with needs: — how to make jobs start the moment their dependencies finish, without waiting for the whole stage to complete, saving up to 50 percent or more of pipeline time. See you in episode 9!