Learn Chef - Core Concepts & Main Architecture

#Introduction

In episode 1 you understood why Chef exists: solving configuration drift, guaranteeing idempotency, and turning infrastructure into code with a pull-based model. The next natural question is: how does all of that work behind the scenes?

Episode 2 answers that by dissecting the main Chef architecture. We will trace the complete flow of a Chef client run — from Ohai collecting node facts, downloading cookbooks and the run_list, compiling recipes, converging state, to reporting the results. Then we will discuss the role of the three main components (Workstation, Chef Infra Server, Nodes) and get to know the nine core components you will use every day.

This is the most important conceptual episode in the early phase of the series. Once you master this architecture map, all the practical episodes that follow are just details on the same framework. Let's start from its heart: the chef-client flow.

#The Chef Client Run Flow

When a node runs chef-client, a sequence of steps takes place in order. Understand this flow well because all Chef troubleshooting comes down to identifying which phase failed.

1. Ohai        ── collect node facts (OS, memory, network, etc.)
2. Download    ── fetch cookbooks & run_list from Chef Infra Server
3. Compile     ── translate recipes into a collection of resources
4. Converge    ── apply each resource (per desired state)
5. Report      ── send run results to the server for auditing

#1. Ohai — Collecting Node Facts

First, chef-client runs Ohai: the tool that collects facts (called attributes) about the node — operating system, kernel version, memory, disk, hostname, IP address, and thousands of other data points. These facts are stored as node attributes that recipes can read later (discussed in episode 5).

#2. Downloading Cookbooks & Run List

The node has a run_list — an ordered list of recipes to apply. Based on this run_list, the node downloads the required cookbooks from the Chef Infra Server, along with all their dependencies.

#3. Compiling Recipes

Ruby recipes are translated into a collection of resources in memory. This phase changes nothing yet — it only builds the "plan" of resources to apply, complete with each one's properties and actions.

#4. Converge — Applying State

The core phase: Chef executes each resource. For every resource, it checks the current state on the node and only acts if there is a difference from the desired state. This is idempotency in action — resources that already match are skipped without doing anything.

#5. Report

Finally, the run results — which resources succeeded, changed, or failed — are sent back to the server to be stored and audited. In the modern ecosystem, these reports can be displayed in Chef Automate.

#The Three Architecture Components

The Chef architecture is divided into three inseparable major roles:

+------------------+      +----------------------+      +----------------+
|  WORKSTATION     | ---> |  CHEF INFRA SERVER   | <--- |     NODES      |
|  writes code     |      |  registry & store    |      |  run           |
|  (author)        |      |  cookbook, data,     |      |  chef-client   |
+------------------+      |  node data, policies |      +----------------+
   upload / knife         +----------------------+          pull & apply

#Workstation

Workstation is the machine where you as a developer write code: creating cookbooks, writing recipes, testing locally, then uploading the results to the server using knife or berks. This is the "author's desk" of infrastructure.

#Chef Infra Server

Chef Infra Server is the central registry and storage warehouse. It stores cookbooks, node data, data bags, environments, roles, and policies. Every node communicates with this server to get the configuration it must apply.

#Nodes

Nodes are the managed machines (managed nodes). On every node runs the chef-client agent, which periodically pulls configuration from the server and applies it. Nodes can be physical servers, VMs, containers, or cloud instances.

#Core Components You Must Know

These are the nine components that will accompany you throughout the series. Understand each one's role:

#Cookbook

The largest unit of Chef code organization. A cookbook contains recipes, attributes, templates, files, libraries, and metadata — all for managing one area of configuration, for example the nginx cookbook.

#Recipe

A Ruby file containing resources to converge. A recipe is the "cooking recipe" — the sequence of resources that describes how a node should be configured.

#Resource

The smallest and most fundamental unit: a single statement about what is desired. Examples: package, service, file, template. Resources will be discussed in depth in episode 4.

#Attribute

Data about the node, both from Ohai (automatic) and from cookbooks (default/override). Attributes give recipes "context" such as OS and application version.

#Run List

The ordered list of recipes (and/or roles) a node must run. The run list is the bridge between a node and its cookbooks.

#Data Bag

Structured global data storage on the server, for example a list of users or application settings. It can be encrypted to store secrets (discussed in episode 7).

#Environment

A logical grouping of nodes, for example development, staging, production. Environments allow different configuration per stage without changing cookbooks.

#Role

A reusable run list template. A webserver role can contain several recipes and be applied to many nodes at once.

#Knife

The main Chef CLI for interacting with the server: knife node list, knife cookbook upload, knife data bag create, and more.

knife node list                       # list all registered nodes
knife cookbook upload nginx           # upload cookbook to the server
knife data bag create admins          # create a new data bag
knife environment show production     # view environment details

#How the Components Work Together

Let's summarize how these components cooperate in one real scenario: a new node wants nginx installed.

1. The workstation creates an nginx cookbook containing a default recipe that declares the package 'nginx' and service 'nginx' resources.
2. The workstation uploads the cookbook to the Chef Infra Server with knife cookbook upload nginx.
3. The node has the run_list recipe[nginx::default].
4. chef-client pulls the run_list and cookbook from the server, compiles the recipe into resources, then converges: installs the nginx package and ensures its service is running.
5. The run results are reported back to the server.

package 'nginx' do
  action :install
end
 
service 'nginx' do
  action [:enable, :start]
end

#Conclusion

In episode 2 we dissected the Chef architecture thoroughly: the five-phase chef client run flow (Ohai, download, compile, converge, report), the role of the three main components (Workstation, Chef Infra Server, Nodes), and the nine core components that form Chef's working language.

Key takeaways:

• The chef-client flow is Ohai → Download → Compile → Converge → Report.
• Workstation writes code, Chef Infra Server stores and distributes, Nodes run chef-client.
• Cookbook contains recipes, recipes contain resources — this is the Chef code hierarchy.
• Run list is the bridge between a node and the cookbooks to be applied.
• Knife is the CLI that connects you to the Chef Infra Server.

Now the architecture map is clear in your head. In the next episode, episode 3, we get our hands dirty: setup and installation — installing Chef Workstation and Chef Client, running chef-client in client, solo, and zero modes, bootstrapping nodes, and creating the first cookbook structure with chef generate cookbook.