Learn Ansible - Cloud Infrastructure Provisioning

#Introduction

After episode 25, where we built the monitoring & observability stack — Prometheus, Grafana, and logging — in this episode we'll move to the environment where most modern infrastructure lives: the cloud.

Let's recall our journey. In episodes 3 and 21 we got to know dynamic inventory for the cloud: the aws_ec2, gcp_compute, and azure_rm plugins that auto-populate the server list based on the cloud state. In episode 19 we saw the collaboration between Ansible and Terraform. Now it's time to cover the provisioning side itself: creating servers, networks, storage, and access in the three largest cloud providers — AWS, Google Cloud, and Azure — using Ansible.

Why is this important? Imagine an engineer opening the AWS Console, clicking "Launch Instance", then choosing options with the mouse to create one server. Now multiply that by 20 servers, 3 environments, and 2 teams. None of those clicks are recorded, can't be reviewed, and the results will almost certainly differ from each other — that's the seed of configuration drift and an uncontrolled cloud bill. On the other hand, if provisioning is written as an Ansible playbook, the whole process can be reviewed in a Pull Request, reproduced at any time, and verified idempotent.

The right roles should also be clarified: Terraform is superior for infrastructure lifecycle and state, while Ansible excels at configuration and application deployment. However, many teams also use Ansible for provisioning — especially when they need one language for the entire stack, or when the volume of infrastructure changes is still small. We'll cover this pattern practically, including how Ansible handles multi-cloud portably.

#Main Discussion

#Getting to Know Ansible's Cloud Collections

Each cloud provider has an official collection managed by the community or vendor:

Install all three:

ansible-galaxy collection install amazon.aws community.aws google.gcp azure.azcollection

#Cloud Credentials & Authentication

Each provider has its own authentication method, and none of them should be stored plaintext in playbooks or inventory:

• AWS — the most common is using the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables, or a profile from ~/.aws/credentials. In playbooks it can be via access_key/secret_key parameters pulled from Vault.
• Google Cloud — a service account JSON file (the service_account_file parameter) or Application Default Credentials (auth_kind: application).
• Azure — the AZURE_SUBSCRIPTION_ID, AZURE_CLIENT_ID, AZURE_SECRET, AZURE_TENANT environment variables (service principal), or an ad_user + password combination.

All playbooks in this episode are assumed to run from localhost (the control node) — provisioning happens via cloud APIs, not SSH:

---
# group_vars/aws.yml
cloud_provider: aws
aws_region: ap-southeast-1
aws_ami: ami-0abcdef1234567890
instance_name: app-01
instance_size: t3.micro

#AWS: Launching an EC2 Instance with amazon.aws.ec2_instance

Let's start with AWS. The ec2_instance module is the successor to the old deprecated ec2 module — it's more idempotent and supports tags well. The following playbook creates two instances at once:

- name: Provision instance EC2
  hosts: localhost
  gather_facts: false
  vars:
    aws_region: ap-southeast-1
    ami_id: ami-0abcdef1234567890
  tasks:
    - name: Launch instance EC2
      amazon.aws.ec2_instance:
        name: "{{ item }}"
        key_name: arman-keypair
        image_id: "{{ ami_id }}"
        instance_type: t3.micro
        region: "{{ aws_region }}"
        security_groups:
          - app-sg
        vpc_subnet_id: subnet-0abc1234
        associate_public_ip_address: true
        tags:
          Environment: production
          Role: app
          ManagedBy: ansible
        state: running
        wait: true
      loop:
        - app-01
        - app-02

Things to note:

• image_id (AMI) is region-specific. An AMI valid in us-east-1 doesn't necessarily exist in ap-southeast-1. Always parameterize and pin to a version — don't use "latest" which can change silently.
• tags aren't just a nicety — tags are how you track costs, group resources, and (crucially!) how the aws_ec2 dynamic inventory plugin selects hosts. An instance without an Environment tag disappears from the cost radar and inventory.
• wait: true makes the playbook wait for the instance to actually be running (and an IP available) before proceeding to the next task — e.g., immediately running a configuration playbook against that instance.

#AWS: Security Groups & VPC

A Security Group is a virtual firewall at the instance level. With Ansible, its rules become declarative and reviewable:

- name: Kelola security group
  hosts: localhost
  gather_facts: false
  vars:
    aws_region: ap-southeast-1
    vpc_id: vpc-0123456789abcdef0
  tasks:
    - name: Buat security group aplikasi
      amazon.aws.ec2_security_group:
        name: app-sg
        description: "Security group untuk aplikasi web"
        region: "{{ aws_region }}"
        vpc_id: "{{ vpc_id }}"
        rules:
          - proto: tcp
            ports: 22
            cidr_ip: "{{ admin_cidr }}"
          - proto: tcp
            ports: 80
            cidr_ip: 0.0.0.0/0
          - proto: tcp
            ports: 443
            cidr_ip: 0.0.0.0/0
        rules_egress:
          - proto: all
            cidr_ip: 0.0.0.0/0

Notice the use of {{ admin_cidr }} for SSH access — restricting SSH to the office/team IP (e.g., 203.0.113.0/24) is one of the cheapest security practices with the biggest impact. VPCs and subnets themselves are managed by the ec2_vpc_net and ec2_vpc_subnet modules with the same idempotent pattern: define cidr_block, state: present, and Ansible maintains the state.

#AWS: S3 Buckets & IAM

S3 buckets are managed by the s3_bucket module — including versioning, encryption, and tags:

- name: Kelola S3 bucket
  hosts: localhost
  gather_facts: false
  tasks:
    - name: Buat bucket penyimpanan backup
      amazon.aws.s3_bucket:
        name: "{{ backup_bucket }}"
        region: "{{ aws_region }}"
        state: present
        versioning: enabled
        encryption: "AES256"
        tags:
          Environment: production
          Purpose: database-backup
 
    - name: Buat user IAM untuk backup service
      amazon.aws.iam_user:
        name: svc-backup
        state: present

IAM (iam_user, iam_role, iam_policy, iam_managed_policy) is the foundation of AWS security. The pattern recommended in the real world: create a dedicated role (e.g., role-backup) with the narrowest policy that only allows s3:PutObject on the backup bucket, then attach that role to instances — instead of storing long-lived access keys inside instances. For RDS, ELB, and Auto Scaling Groups, the relevant modules are rds_instance, elb_classic_lb/elb_application_lb, and autoscaling_group (all in community.aws) — the pattern is identical: declarative, idempotent, state-based.

#Google Cloud: Compute Engine & Networking

For GCP, the google.gcp collection provides modules with a resource-based pattern (one module per resource). Example of creating a Compute Engine instance:

- name: Provision instance Compute Engine
  hosts: localhost
  gather_facts: false
  vars:
    gcp_project: my-company-project
    gcp_zone: asia-southeast1-a
    gcp_sa_file: /opt/keys/gcp-sa.json
  tasks:
    - name: Buat instance app-01
      google.gcp.gcp_compute_instance:
        name: app-01
        project: "{{ gcp_project }}"
        zone: "{{ gcp_zone }}"
        machine_type: e2-small
        auth_kind: serviceaccount
        service_account_file: "{{ gcp_sa_file }}"
        disks:
          - auto_delete: true
            boot: true
            initialize_params:
              source_image: "projects/ubuntu-os-cloud/global/images/family/ubuntu-2404-lts"
        network_interfaces:
          - network: "projects/{{ gcp_project }}/global/networks/default"
            access_configs:
              - name: External NAT
                type: ONE_TO_ONE_NAT
        tags:
          items: ["http-server"]
        metadata:
          ssh-keys: "arman:{{ vault_ssh_pubkey }}"
        state: present

Some GCP-specific things:

• Zone vs region — instances live in a zone (asia-southeast1-a), while regional resources (VPC, firewall) live in a region. Mixing the two is the most common source of confusion.
• source_image uses an image family — similar to the "latest" concept, but stabilized per family (e.g., ubuntu-2404-lts), so you get OS updates without changing the playbook.
• auth_kind: serviceaccount + service_account_file — the official authentication for automation. The service account JSON file must be Vaulted.

VPC networks (gcp_compute_network) and firewall rules (gcp_compute_firewall) follow the same pattern, while GCS buckets use gcp_storage_bucket and Cloud SQL uses gcp_sql_instance. The gcp_compute dynamic inventory plugin (episode 21) then automatically reads all these instances for configuration playbooks.

#Azure: Virtual Machines & Networking

Azure uses the azure.azcollection collection with modules prefixed azure_rm_. Example of creating a Virtual Machine:

- name: Provision Virtual Machine di Azure
  hosts: localhost
  gather_facts: false
  vars:
    azure_rg: rg-app
    azure_location: southeastasia
  tasks:
    - name: Buat VM app-01
      azure.azcollection.azure_rm_virtualmachine:
        resource_group: "{{ azure_rg }}"
        name: app-01
        vm_size: Standard_B2s
        admin_username: arman
        ssh_public_key_file: ~/.ssh/id_ed25519.pub
        image:
          offer: UbuntuServer
          publisher: Canonical
          sku: "22.04-LTS"
          version: latest
        network_interfaces:
          - name: app-01-nic
            virtual_network:
              name: vnet-app
            subnet:
              name: snet-app
            security_group:
              name: nsg-app
        state: present

Notice that Azure VMs require explicit network infrastructure from the start: virtual_network, subnet, and security_group are all referenced (and can be auto-created) by the VM module. Those network resources themselves are managed by the azure_rm_virtualnetwork, azure_rm_subnet, and azure_rm_networksecuritygroup modules. Storage accounts use azure_rm_storageaccount and Azure SQL uses azure_rm_sqlserver. The "resource managed by one module, referenced by another" pattern is consistent throughout the Azure collection.

#Multi-Cloud Patterns: Abstraction & Portability

Now the big question: what if your company uses more than one cloud — or wants to be able to switch? Ansible offers two complementary patterns:

1. Abstraction variables + conditional when. Define a single cloud_provider variable, then run cloud-specific tasks conditionally. This is the simplest and clearest pattern:

- name: Provision instance sesuai cloud provider
  hosts: localhost
  gather_facts: false
  vars:
    cloud_provider: aws
    instance_name: app-01
    instance_size:
      aws: t3.micro
      gcp: e2-small
      azure: Standard_B2s
  tasks:
    - name: Launch instance di AWS
      amazon.aws.ec2_instance:
        name: "{{ instance_name }}"
        image_id: "{{ aws_ami_id }}"
        instance_type: "{{ instance_size[cloud_provider] }}"
        region: "{{ aws_region }}"
      when: cloud_provider == "aws"
 
    - name: Launch instance di Google Cloud
      google.gcp.gcp_compute_instance:
        name: "{{ instance_name }}"
        project: "{{ gcp_project }}"
        zone: "{{ gcp_zone }}"
        machine_type: "{{ instance_size[cloud_provider] }}"
      when: cloud_provider == "gcp"
 
    - name: Launch VM di Azure
      azure.azcollection.azure_rm_virtualmachine:
        resource_group: "{{ azure_rg }}"
        name: "{{ instance_name }}"
        vm_size: "{{ instance_size[cloud_provider] }}"
      when: cloud_provider == "azure"

2. Dictionary lookup. Values that are "mappings per provider" (instance size, AMI, zone) are stored as dictionaries, selected with {{ instance_size[cloud_provider] }}. This keeps the playbook DRY for common values, while provider-specific tasks stay explicit with when.

But there's an important warning. Full abstraction between clouds (one playbook for all providers) is indeed tempting, but there's a trade-off:

The most common multi-cloud pattern in the real world isn't actually abstraction, but dynamic inventory combination: run the same playbook against host groups from aws_ec2, gcp_compute, and azure_rm simultaneously, with per-provider group_vars carrying the configuration differences (package manager, daemon, paths). This leverages Ansible's native strength — one playbook, many host types — without forcing uniformity on provisioning modules.

#Common Pitfalls

1. Hardcoded AMI / image IDs

AMIs differ per region; GCP images per family; Azure SKUs per publication. Hardcoded values = a playbook that breaks when the region changes. Parameterize and version.

2. Cloud credentials committed to Git

An AWS access key or GCP service account in a repository is a time bomb. Always Vault or secret store them — and remember: Git never forgets (episode 14).

3. Provisioning without wait

Without wait: true, the playbook proceeds to configuration tasks (SSH) before the instance is ready — the result is Connection refused. For S3/buckets that need propagation, use the retry module (until + retries from episode 9).

4. Forgetting tags for billing & inventory

Instances without tags are hard to track for cost and hard to select with dynamic inventory plugins. Set a tag standard (Environment, Team, CostCenter) from the first provisioning.

5. Mixing regions/zones

VPC created in region A, instance in a zone of region B — the network never connects. Always declare region/zone/location explicitly and consistently.

6. Ignoring collection Python dependencies

Especially Azure: forgetting to run pip install -r requirements-azure.txt makes all azure_rm_* modules fail. Document these requirements in the project README.

7. Over-abstraction

A single playbook trying to unify all clouds often produces nested when conditions that are hard to read, with features cut off. Abstract just enough; leverage the differences.

#Conclusion

In this episode we've covered cloud provisioning with Ansible: the amazon.aws/community.aws collections for EC2, Security Groups, VPC, S3, and IAM; google.gcp for Compute Engine, networking, and firewalls; azure.azcollection for Virtual Machines, Virtual Networks, and Network Security Groups; and multi-cloud patterns in the form of abstraction variables and conditional when — complete with the trade-offs you should understand before deciding how far to take abstraction.

The key: cloud provisioning written as code makes infrastructure auditable, reproducible, and idempotent — no longer dependent on console clicks and human memory. Combined with dynamic inventory from episode 21, you now have the full cycle: provisioning → configuration → monitoring.

In episode 27, we'll secure everything we've built: Security Hardening & Compliance Automation — implementing CIS benchmarks, SSH and firewall hardening, automated patch management, and compliance as code with STIG and tools like OpenSCAP and Lynis. Keep your enthusiasm up!