Learn Chef - Cloud & IaC Integration

#Introduction

In episode 13 you secured communication between Chef components with keys and certificates. Now it is time to take that architecture to the cloud. In episode 14 we discuss Chef integration with cloud and IaC — managing AWS, GCP, and Azure instances, connecting Chef with Terraform and cloud-init, and applying the patterns of automated bootstrap, golden images, and immutable infrastructure.

The goals of this episode:

• Manage cloud instances through knife plugins.
• Integrate Terraform and cloud-init for provisioning.
• Bootstrap nodes automatically in CI/CD.
• Understand golden images and immutable infrastructure.

#Managing Cloud Instances via Knife

Knife plugins such as knife-ec2, knife-google, and knife-azure let you create instances directly from the CLI while also bootstrapping them into Chef nodes:

knife ec2 server create \
  -I ami-0abcdef1234567890 \
  -f t3.micro \
  --ssh-key arman-key --ssh-user ubuntu \
  -Z ap-southeast-1a \
  --run-list 'role[app-server]' \
  --tags Environment=staging

#Provisioning via Recipe

Another alternative is provisioning directly from a recipe, for example by calling a cloud SDK within a Ruby library. However, this pattern is rarely used because it mixes orchestration with configuration. The more common and recommended pattern: bootstrap at the end of the provisioning pipeline — let Terraform create the infrastructure, then have nodes connect to Chef themselves.

#Terraform and cloud-init Integration

Terraform builds the infrastructure (network, instances, disks); cloud-init executes user-data the moment an instance first boots. This is where Chef gets bootstrapped:

#!/bin/bash
set -euxo pipefail
apt-get update -y
curl -L https://omnitruck.chef.io/install.sh | sudo bash -s -- -v 18.5
sudo mkdir -p /etc/chef/trusted_certs
sudo cp /tmp/chef-server.crt /etc/chef/trusted_certs/
sudo tee /etc/chef/client.rb > /dev/null <<EOF
chef_server_url "https://chef.example.com/organizations/devops"
validation_client_name "devops-validator"
validation_key "/etc/chef/validation.pem"
node_name "web-$(hostname -s)"
environment "staging"
EOF
sudo chef-client -j /tmp/first-boot.json

The first-boot.json file contains the initial run list:

{
  "run_list": ["role[base]", "role[app-server]"]
}

Terraform sends the script above via the user_data argument. As soon as the instance boots, cloud-init runs it, chef-client gets installed, and the node registers with the Infra Server with no human involvement.

#Automated Bootstrap in CI/CD

The automated bootstrap pattern in a pipeline:

1. Provision — terraform apply creates a new instance.
2. Bootstrap — the instance bootstraps itself via cloud-init at boot.
3. Converge — chef-client runs the run list from the Infra Server.
4. Verify — InSpec scans the node and reports are sent to Automate.
5. Promote — if it passes, the node joins the production pool.

terraform apply -auto-approve
sleep 60
chef-client

#Golden Images and Immutable Infrastructure

A golden image is a prepared base image: chef-client installed, trust certificates ready, basic tooling in place. A new node only needs a thin user-data script to finish the bootstrap.

Immutable infrastructure means an instance is never changed after it is born — every change produces a new image, and then the old instance is replaced. This pattern pairs well with golden images: Chef is used when building the image, not while the instance is running.

#Conclusion

In this episode 14 you understood Chef's integration with cloud and IaC: creating instances through knife plugins, Terraform provisioning combined with cloud-init for automated node bootstrap, the bootstrap pipeline in CI/CD, and the concepts of golden images and immutable infrastructure.

Key takeaways:

• Knife plugins speed up manual instance bootstrap.
• Terraform + cloud-init is the most common provisioning pair.
• Automated bootstrap removes manual steps when an instance is born.
• Golden images shorten the lifecycle of new nodes.
• Immutable infra replaces instances instead of changing them.

In the next episode, episode 15, we polish our working habits: Best Practices — designing idempotent and modular cookbooks, correct dependency metadata, versioning, and test-driven infrastructure with InSpec and Test Kitchen. See you in episode 15!