Learn GitLab CI/CD - Pipeline Modularization with include & Child Pipelines

#Introduction

In the previous episode 10 we covered how parallel: matrix and parallel: 5 split work across many parallel runners so pipeline duration can be cut dramatically. Now imagine two years later: the repository is held by many teams, and the .gitlab-ci.yml file at the repository root has swollen to thousands of lines with jobs copying each other's configuration. Every team copy-pastes the same blocks, and one small change to build config means editing it while thinking about its impact on dozens of jobs across the entire repository.

This situation is exactly like a giant notebook shared by the whole class: every time one student wants to write, others have to wait, and everyone has to read the same pages just to find their own section. In the real world, DevOps teams don't work that way. They split large files into modules, store common configuration in a centralized place, and orchestrate small pipelines each responsible for their own domain.

This episode covers GitLab's two main tools for solving that problem: the include keyword for code reusability, and the parent-child pipelines and multi-project pipelines techniques for splitting large pipelines into independent sub-pipelines. This is the foundation of enterprise pipeline architecture we'll keep using in the upcoming episodes.

#Main Discussion

#Why a Monolithic .gitlab-ci.yml File Becomes a Problem

Before discussing the solution, let's understand the root problem first. A monolithic pipeline file isn't just "long" — it creates four concrete problems in real teams:

1. Merge request conflicts. Two teams editing the same file almost certainly collide on git merge, and resolving pipeline conflicts is one of the most unpleasant jobs.
2. No cross-project reuse. The same Docker build or deploy config is rewritten in every repository, and when the pattern changes, every repository has to change one by one.
3. Wide-reaching changes. One small edit at the top of the file can accidentally change the behavior of dozens of jobs below it.
4. Hard to read and audit. Reviewers and new members have to understand the whole pipeline just to approve one small change.

GitLab's solution to these problems is the include keyword, which lets a pipeline be built from many YAML files — much like how modern programming languages split code into modules and functions.

#include: local: Modules Within the Same Repository

include: local imports YAML files from the repository being processed. This is the most basic pattern for splitting a large .gitlab-ci.yml into per-domain files.

include:
  - local: /ci/build.yml
  - local: /ci/test.yml
  - local: /ci/deploy.yml
 
stages: [build, test, deploy]

build-backend:
  stage: build
  image: node:20-alpine
  script:
    - npm ci
    - npm run build

A local path always starts with / (relative to the repository root) or is relative to the file including it. Included files can contain job definitions, stages, variables, or anything valid in .gitlab-ci.yml.

#include: project: Cross-Repository Config Reuse

The local limitation is that it only works within one repository. But linting, build, or deploy configuration is usually identical across many projects. include: project imports YAML files from another GitLab repository — within the same organization or even from public projects.

include:
  - project: "my-org/ci-templates"
    ref: v2.4.0
    file: /templates/build.yml
  - project: "my-org/ci-templates"
    ref: v2.4.0
    file: /templates/test.yml

Note the ref and file pair: ref selects a branch, tag, or commit from the source project. Binding includes to a tag (v2.4.0) instead of the main branch is best practice — this is how an internal company template gets versioned with semantic versioning, just like a software library.

#include: template: Official GitLab Templates

include: template imports official CI templates maintained by the GitLab team. These templates cover security jobs, SAST, dependency scanning, and various programming languages.

include:
  - template: Jobs/SAST.gitlab-ci.yml
  - template: Security/Container-Scanning.gitlab-ci.yml

Templates are the fastest way to adopt GitLab features without writing anything. We'll dissect these security templates in depth in episode 12.

#include: remote: Importing from HTTP/S URLs

include: remote fetches YAML files from an external URL. This is useful for sharing configuration across organizations, for example community-maintained public templates.

include:
  - remote: "https://example.com/team/pipeline-base.yml"

#extends: Configuration Inheritance Without Copy-Paste

In addition to include, GitLab provides the extends keyword to inherit configuration from other jobs in the same file. This replaces the now-deprecated YAML anchors in modern versions.

.base-job:
  image: node:20-alpine
  variables:
    NODE_ENV: production
  before_script:
    - npm ci
 
build-app:
  extends: .base-job
  stage: build
  script:
    - npm run build

Notice that a job starting with a dot (.base-job) is not executed by GitLab — it's only used as a template. extends performs a config merge: values not overridden are inherited, while values written in the derived job override the parent's. This is the most idiomatic way to avoid copy-paste.

#Parent-Child Pipelines: Orchestration Within One Repository

When a repository grows into a monorepo — say, containing frontend, backend, and docs folders — a single giant pipeline ties all teams to a shared configuration. Parent-child pipelines break that up: a parent pipeline triggers child sub-pipelines, each with its own YAML file and lifecycle.

stages: [pre, build, deploy]
 
build-children:
  stage: pre
  trigger:
    include:
      - local: /frontend/child-ci.yml
      - local: /backend/child-ci.yml
    strategy: depend

The trigger keyword makes a job act as a trigger for another pipeline, instead of executing a script. With strategy: depend, the parent pipeline waits for the child pipelines to finish before continuing to the next stage — suitable when the monorepo build must finish before deploy. Each child pipeline runs under its parent pipeline's umbrella and can use predefined variables like CI_PROJECT_DIR for its own folders.

#Multi-Project Pipelines: Connecting Pipelines Across Repositories

The opposite of a child pipeline (still in the same repository), a multi-project pipeline triggers pipelines in separate repositories. This is the backbone of the downstream pipeline pattern: repository A (e.g. a shared library) finishes building, then triggers tests in repository B (the app consuming that library).

trigger-deploy:
  stage: deploy
  trigger:
    project: my-org/api-gateway
    branch: main
    strategy: depend

To trigger a pipeline in another project, GitLab uses the CI_JOB_TOKEN from the triggering project — meaning the target project must grant access permission through the Pipeline trigger feature in project settings. Downstream pipeline results can be monitored directly from the upstream pipeline in the GitLab UI.

#When to Use Which

#Closing

In this episode we've covered how to break up monolithic pipelines: the four forms of include — local, project, template, and remote — with examples of each; the extends keyword for inheriting configuration between jobs; parent-child pipelines with trigger: include ideal for monorepos; and multi-project pipelines with trigger: project for connecting pipelines across repositories.

The core of this episode: a pipeline is not one giant file, but an orchestration of small, independent modules. With modularization, teams can move fast, templates can be versioned, and each pipeline part is responsible for its own domain.

In the next episode 12 we enter PHASE 5: Native GitLab DevSecOps — enabling SAST, Secret Detection, Dependency Scanning, and Container Scanning with just a few lines of include: template, then reading the results from the Security Center. See you there!