Learn Ansible - CI/CD Pipeline Integration & IaC Paradigm

#Introduction

After episode 18, where we covered maintaining Ansible code quality with ansible-lint and testing it in isolation with Molecule, you now have powerful weapons to ensure infrastructure code is safe before it has impact. But there's one unanswered question: who runs all of that, and when?

If the answer is "we run it manually on each person's laptop", then we're only halfway there. In the real world, healthy infrastructure teams don't leave quality up to individual discipline, because humans always forget. The correct answer is a CI/CD pipeline: a machine that runs linting, testing, and deployment automatically every time the code changes.

Episode 19 will bring you to understand the Infrastructure as Code (IaC) and GitOps paradigms in the Ansible context. We'll build pipelines with GitHub Actions and GitLab CI, discuss how to handle secrets in CI safely, and finally learn the collaboration between Ansible and Terraform — two automation giants that actually complement each other, not compete. This is material you'll use almost every day as an infrastructure engineer.

#Main Discussion

#Why Does Ansible Need a CI/CD Pipeline?

Let's start from the most fundamental question. What changes if we run ansible-playbook from a laptop?

Problem one: consistency. A playbook run from laptop A can produce different results than laptop B, only because of different Ansible, Python, or collection versions. We'll thoroughly dissect this problem in episode 20 with the execution environments concept.

Problem two: no audit trail. When an engineer runs ansible-playbook -i production.yml deploy.yml from their laptop at 2 AM, there's no record of who did it, what changes were triggered, and whether there was a prior review stage.

Problem three: no quality gate. Without a pipeline, nothing forces code through ansible-lint and Molecule before it reaches production.

With a CI/CD pipeline, all these problems are solved. Code changes → CI runs quality → code passes → deployment runs → everything is recorded. This is the core of the GitOps paradigm: Git is the single source of truth. No configuration changes on a server unless that change originates from the Git repository.

#The GitOps Flow for Ansible Infrastructure

The most common GitOps scenario for Ansible looks like this:

1. An engineer creates a new branch for a playbook/role change.
2. Opens a Pull Request (PR). The CI pipeline runs immediately: ansible-lint + Molecule test.
3. Another engineer does a code review in the PR.
4. The PR is merged into the main branch.
5. A trigger from that merge runs the deployment pipeline to staging, then production.

The key is point 5: deployment only happens when there's a merge to main, not when someone types a command manually. All processes are recorded in Git history and pipeline logs.

#CI/CD Pipeline with GitHub Actions

GitHub Actions is the most popular choice for projects whose repositories are on GitHub. Workflows are defined as YAML in the .github/workflows/ directory. Here's a complete workflow example running lint → test → deploy with automatic triggers on PRs and on merges to main:

name: Ansible CI/CD
 
on:
  pull_request:
    paths:
      - "ansible/**"
  push:
    branches: [main]
    paths:
      - "ansible/**"
 
jobs:
  lint:
    name: Lint playbooks & roles
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
 
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
 
      - name: Install ansible-lint
        run: pip install ansible-lint
 
      - name: Run ansible-lint
        run: ansible-lint
 
  molecule:
    name: Molecule test (Docker)
    runs-on: ubuntu-latest
    needs: lint
    steps:
      - uses: actions/checkout@v4
 
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
 
      - name: Install Molecule
        run: pip install "molecule[docker]"
 
      - name: Run molecule test
        run: molecule test
 
  deploy:
    name: Deploy to production
    runs-on: ubuntu-latest
    needs: [lint, molecule]
    if: github.ref == 'refs/heads/main'
    steps:
      - uses: actions/checkout@v4
 
      - name: Setup SSH key
        run: |
          mkdir -p ~/.ssh
          echo "${{ secrets.DEPLOY_SSH_PRIVATE_KEY }}" > ~/.ssh/id_ed25519
          chmod 600 ~/.ssh/id_ed25519
          ssh-keyscan github.com >> ~/.ssh/known_hosts
 
      - name: Install Ansible & collections
        run: |
          pip install ansible-core
          ansible-galaxy collection install -r requirements.yml
 
      - name: Run ansible-playbook
        run: |
          ansible-playbook \
            -i inventory/production.yml \
            --vault-password-file <(echo "${{ secrets.VAULT_PASSWORD }}") \
            playbooks/deploy.yml

Let's dissect the workflow above:

• Triggers (on) are defined so the pipeline runs on two events: every PR changing files in the ansible/ directory, and every push to the main branch. This paths trigger is important so the pipeline doesn't waste resources on changes that don't touch Ansible code.
• Job lint runs ansible-lint from the project root. If there are error violations, this job fails and the PR can't be merged.
• Job molecule runs molecule test with Docker. It waits for the lint job to finish (needs: lint) so the test order is structured.
• Job deploy only runs when github.ref == 'refs/heads/main' — that is, after the PR is merged, not when the PR is opened. This is exactly the GitOps pattern we discussed.
• Secrets are pulled from GitHub Secrets via ${{ secrets.SECRET_NAME }}. The SSH private key is written to ~/.ssh with the correct permission, and the Vault password is injected as a password file.

#Alternative: CI/CD Pipeline with GitLab CI

For teams using GitLab, the same flow is built with the .gitlab-ci.yml file. GitLab CI uses the stages and jobs concepts, and can run DinD (Docker in Docker) for Molecule needs:

---
stages:
  - lint
  - test
  - deploy
 
variables:
  PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip"
 
cache:
  paths:
    - .cache/pip
 
lint:
  stage: lint
  image: python:3.12-slim
  script:
    - pip install --no-cache-dir ansible-lint
    - ansible-lint
  only:
    - merge_requests
    - main
 
test:
  stage: test
  image: docker:27
  services:
    - docker:27-dind
  variables:
    DOCKER_HOST: tcp://docker:2376
    DOCKER_TLS_CERTDIR: /certs
  script:
    - pip install --no-cache-dir "molecule[docker]"
    - molecule test
  only:
    - merge_requests
    - main
 
deploy:
  stage: deploy
  image: python:3.12-slim
  script:
    - apk add --no-cache openssh-client
    - pip install --no-cache-dir ansible-core
    - ansible-galaxy collection install -r requirements.yml
    - echo "$VAULT_PASSWORD" > .vault-pass
    - chmod 600 .vault-pass
    - ansible-playbook -i inventory/production.yml --vault-password-file .vault-pass playbooks/deploy.yml
  only:
    - main
  when: manual

Notice several differences and similarities with GitHub Actions:

• Stages in GitLab CI are similar to job ordering in GitHub Actions; jobs in the next stage wait for the previous stage to finish.
• The test job uses the Docker-in-Docker service so the docker and molecule commands can run inside the runner.
• The deploy job is defined with only: main and when: manual, meaning it waits for a manual approval from an engineer before actually running the production deployment. This is a very common pattern for production: full automation up to staging, but production needs human approval.
• The VAULT_PASSWORD secret is taken from GitLab CI/CD Variables.

#Brief Comparison: GitHub Actions vs GitLab CI

#Handling Secrets in CI Safely

Secrets are the most sensitive part of a pipeline. The best patterns you need to apply:

1. Store secrets in the CI platform, not in the repository. GitHub calls them Secrets, GitLab calls them CI/CD Variables.
2. Use Ansible Vault for data that must be in the repository (e.g., encrypted credential files), and store only the vault password in the CI platform.
3. Limit secret access per environment. On GitHub, create a separate production environment with protection rules; production secrets must not be visible to PR jobs.
4. Never log secrets. Avoid verbose modes in CI that could print variables, and don't put secrets as arguments visible in the process list.

#Ansible & Terraform Collaboration: Clear Role Division

Now we get to the section many engineers love: how Terraform and Ansible work together. There's often confusion about whether these two tools compete. The answer: no. They solve different problems, and in a healthy production team, both are used together.

The easiest-to-remember role division:

A fitting analogy: Terraform is like the contractor who builds the building (creating the foundation, structure, electrical installation), while Ansible is like the interior team that arranges the room's contents (installing furniture, organizing decor, making sure everything is functional). Each is an expert in their domain, and the best results come when both work in sequence.

#The Terraform → Ansible Workflow

The most common collaboration flow is provision first, configure after:

1. Terraform apply — Creates the infrastructure: VPC, subnet, security group, and VM instances.
2. Terraform output — Exports important information, most importantly the IP addresses of the newly created resources.
3. Ansible — Takes those IPs and runs configuration management & application deployment to the newly born servers.

The key to this collaboration is how Ansible gets the IPs from Terraform. There are several ways, and we'll look at the simplest and most widely used.

First, define the outputs in the Terraform file. Example for an AWS instance resource:

output "web_public_ip" {
  description = "Public IP dari instance web"
  value       = aws_instance.web.public_ip
}
 
output "web_private_ip" {
  description = "Private IP dari instance web"
  value       = aws_instance.web.private_ip
}

After terraform apply finishes, you can view the output results:

terraform output

The displayed output:

web_private_ip = "10.0.1.10"
web_public_ip = "203.0.113.10"

For consumption by scripts/CI, use the JSON format:

terraform output -json

{
  "web_public_ip": {
    "sensitive": false,
    "type": "string",
    "value": "203.0.113.10"
  }
}

#Connecting Terraform Output to Ansible

There are three common approaches to connect Terraform output with Ansible:

1. Extra Vars (simplest). Grab the IP as an extra var when running the playbook:

ansible-playbook -i inventory/production.yml \
  -e "web_public_ip=$(terraform output -raw web_public_ip)" \
  -e "web_private_ip=$(terraform output -raw web_private_ip)" \
  playbooks/deploy-app.yml

Inside the playbook, those IPs can be used with add_host so the newly created server becomes part of the runtime inventory:

---
- name: Registrasi server baru ke inventory runtime
  hosts: localhost
  gather_facts: false
  tasks:
    - name: Tambahkan web server yang baru dibuat
      ansible.builtin.add_host:
        name: web01
        ansible_host: "{{ web_public_ip }}"
        ansible_user: ubuntu
        groups:
          - webservers
 
- name: Konfigurasi & deploy aplikasi ke webservers
  hosts: webservers
  become: true
  tasks:
    - name: Install Nginx
      ansible.builtin.apt:
        name: nginx
        state: present
        update_cache: true
 
    - name: Deploy aplikasi
      ansible.builtin.copy:
        src: files/app/
        dest: /var/www/html
        mode: "0644"

2. Generate an inventory file. Save the output to an inventory file dynamically, suitable for scenarios needing a permanent inventory:

cat > inventory/generated.yml <<EOF
all:
  children:
    webservers:
      hosts:
        web01:
          ansible_host: "$(terraform output -raw web_public_ip)"
          ansible_user: ubuntu
EOF

3. Dynamic inventory plugin. In episode 21, we'll discuss dynamic inventory in depth, including integrating data directly from cloud providers without having to manually export output.

#Pitfall: Execution Order and State Drift

Terraform–Ansible collaboration has a few traps you must understand:

1. Ansible must not run before the infrastructure is ready. If ansible-playbook runs before terraform apply finishes, the IPs used don't exist yet and SSH connections will fail. Make sure in the pipeline that the Ansible job is defined after the Terraform job finishes (needs: in GitHub Actions or a later stage in GitLab CI).

2. Don't use terraform output -raw without a valid state. Output only reflects Terraform's last state. If the state is stale (e.g., an instance deleted outside Terraform), the produced IPs are useless.

3. SSH key security. New servers usually only have the key injected during provisioning. Make sure Ansible uses the correct key via ansible_ssh_private_key_file or the SSH agent in CI.

4. Idempotency still applies. Re-running the Ansible playbook on an already-configured server must still produce ok, not repeated changed — exactly the principle we've learned since episode 5 and tested with Molecule in episode 18.

#Complete Workflow in the Pipeline

When combined with the pipeline from the beginning of the episode, the complete Infrastructure as Code workflow in a modern organization looks like this:

1. An engineer changes Terraform code and/or Ansible code in one PR.
2. CI runs terraform fmt/terraform plan for infrastructure changes, and ansible-lint + Molecule for Ansible changes.
3. The PR is reviewed then merged into main.
4. The deploy pipeline runs terraform apply (creating/changing infrastructure).
5. The pipeline takes the Terraform output and runs ansible-playbook against the new/changed servers.
6. All results are recorded in pipeline logs for audit.

This flow eliminates "the person executing from a laptop", closes the human gap, and makes the entire infrastructure truly as code.

#Conclusion

In episode 19, we learned that Ansible automation only achieves its full value when integrated into a CI/CD pipeline. We understood the GitOps paradigm: Git as the single source of truth, where changes only take effect after being merged into the main branch. We also built complete pipelines with GitHub Actions and GitLab CI, learned how to handle secrets safely, and mapped the clear role division between Terraform (infrastructure provisioning) and Ansible (configuration management & application deployment).

Key points to take home:

• CI/CD pipelines ensure quality and consistency without depending on individual discipline.
• Deployment triggers must be clear: merge to main for staging, manual approval for production.
• Secrets are stored in the CI platform, never in the repository or logs.
• Terraform answers "what should exist", Ansible answers "how it's configured".
• Terraform output can be connected to Ansible via extra vars, generated inventory, or dynamic inventory.

In episode 20, we'll cover one of the most anticipated topics: Modern Execution Environments. We'll trace the "works on my machine" problem that haunts large teams, then learn to use ansible-builder to create container images containing a consistent Ansible environment, ansible-navigator to run container-based automation, and get to know AWX/Ansible Automation Platform for enterprise scale with RBAC, job scheduling, and audit logging. Keep your enthusiasm up!