Learn Terraform - State Management (Local vs Remote State)

#Introduction

After discussing in episode 4 how input variables make configurations flexible across environments, local values eliminate expression repetition, and output values share work results — in this episode we'll discuss the topic that is most often misunderstood yet most determines a team's stability: state management.

All the examples in the previous episodes, without you realizing it, have been building a file called terraform.tfstate every time terraform apply runs. This file is what tells Terraform that the i-0abcd1234efgh5678 instance in AWS "belongs to" the aws_instance.web resource in your code. Without this file, Terraform would be confused: should it create a new instance? Or is this existing instance already managed?

Why is this topic crucial in the real working world? Because the majority of IaC incidents happen not because of syntax errors, but because of state: state files lost, forcing teams to rebuild entire infrastructure; state files conflicting because two people ran terraform apply at the same time; or state files leaking because they were committed to Git along with database passwords inside. Understanding state — what it contains, how to secure it, and how to store it remotely — is a mandatory prerequisite before you touch any team's infrastructure.

#Main Discussion

#Why Does Terraform Need a State File?

Terraform is declarative: you write the desired end state, and Terraform determines the steps to get there. But to do that, Terraform must know the current condition. The state file is that source of knowledge.

Think of state like an accounting ledger or purchase receipts. Your HCL code is the "shopping plan", while state is proof of "what has been bought, with what transaction IDs". If you buy the same item twice without recording it, you end up with duplicates and waste. Exactly like that: without state, every terraform apply would create a new EC2 instance again — because Terraform doesn't know that the instance in question was already created.

Technically, state stores the mapping between HCL code and the real resource IDs in the cloud:

This mapping is why Terraform can do three important things: detect drift (compare cloud condition vs state), accurate planning (only change resources that changed), and precise destruction (delete the correct resources, not guess).

#The State File Structure (JSON)

The terraform.tfstate file is in JSON format. You don't need to read it manually for day-to-day work, but understanding its structure helps remove the "fear" of this file:

{
  "version": 4,
  "terraform_version": "1.9.5",
  "resources": [
    {
      "module": "",
      "mode": "managed",
      "type": "aws_instance",
      "name": "web",
      "provider": "provider[\"registry.terraform.io/hashicorp/aws\"]",
      "instances": [
        {
          "schema_version": 1,
          "attributes": {
            "id": "i-0abcd1234efgh5678",
            "ami": "ami-0c55b159cbfafe1f0",
            "instance_type": "t3.micro",
            "tags": {
              "Name": "web-server"
            }
          }
        }
      ]
    }
  ]
}

Notice the correspondence with the HCL code: the "type": "aws_instance" + "name": "web" block exactly mirrors resource "aws_instance" "web", and "id" is the bridge to the real object in AWS. This file also stores all resource attributes — including ones that may be sensitive.

#The Dangers of Local State for an Engineering Team

When you work alone, local state in the project folder feels convenient. But once a team starts collaborating, the local terraform.tfstate becomes a time bomb. There are three main problems:

Let's break down the third problem deeper with a real scenario:

That's why professional engineering teams never store state locally. The solution is a remote state backend.

#Remote State Backend & State Locking

A backend is where Terraform stores state centrally. With a backend, state lives in one shared location (e.g. an S3 bucket), so all team members read the same state — and more importantly, a consistent one.

However, storing state in one place isn't enough. There's still a gap: two people can read the same state then apply concurrently. That's where state locking comes in — a locking mechanism that ensures only one terraform apply/plan holds state access at a time.

Think of it like a locked public toilet: a second person trying to enter while someone is still inside will wait (or be immediately rejected with an error message), rather than entering and messing things up. Without a lock, two people could enter at once and create chaos.

#AWS S3 + DynamoDB Locking Backend

The most popular backend configuration in the AWS ecosystem is the combination of S3 (for storing state) + DynamoDB (for state locking):

terraform {
  backend "s3" {
    bucket         = "myapp-tfstate-bucket"
    key            = "production/network/terraform.tfstate"
    region         = "ap-southeast-1"
    encrypt        = true
    dynamodb_table = "myapp-tfstate-lock"
  }
}

Attribute explanation:

• bucket + key — the object location of the state. A neat key pattern separates environments and components, e.g. production/network/ vs staging/app/.
• encrypt = true — enforces server-side encryption on S3 (one extra security layer for sensitive data in state).
• dynamodb_table — the name of the DynamoDB table that acts as the lock table. Terraform creates a lock item in this table while an operation runs and deletes it when done.

#Google Cloud Storage (GCS) Backend

For teams in the Google Cloud ecosystem, the GCS backend is the primary choice:

terraform {
  backend "gcs" {
    bucket = "myapp-tfstate-bucket"
    prefix = "production/network"
  }
}

GCS handles state locking natively through object versioning — no separate DynamoDB table needed like in AWS. Just enable Object Versioning on the bucket, and Terraform uses GCS preconditions as the locking mechanism.

#Other Commonly Used Backends

#Local State vs Remote State

Here's a concise comparison to reinforce the architectural decision:

#Inspecting State: state list and state show

Even though the state file shouldn't be read manually, Terraform provides commands to introspect it safely:

terraform state list

aws_instance.web
aws_s3_bucket.artifacts
data.aws_ami.ubuntu

terraform state show aws_instance.web

# aws_instance.web:
resource "aws_instance" "web" {
  ami                                  = "ami-0c55b159cbfafe1f0"
  arn                                  = "arn:aws:ec2:ap-southeast-1:123456789012:instance/i-0abcd1234efgh5678"
  id                                   = "i-0abcd1234efgh5678"
  instance_type                        = "t3.micro"
  tags                                 = {
    "Name" = "web-server"
  }
  ...
}

These two commands are very useful when verifying that the state mapping is correct before operations like terraform destroy or refactoring (which we'll discuss further in episode 8).

#The Golden Rule: Never Commit State to Git

Because state contains sensitive data and is machine-specific, the rule is simple: never commit terraform.tfstate, terraform.tfstate.backup, or any *.tfstate to Git. Add them to .gitignore:

# Terraform state (local, because a remote backend is already in use)
*.tfstate
*.tfstate.backup
*.tfstate.lock.info
 
# Secrets often confused with state
*.tfvars

#Common Mistakes in State Management

#Conclusion

In this episode 5 we discussed state management thoroughly: why Terraform needs the terraform.tfstate file as a mapping between HCL code and real IDs in the cloud, the JSON structure behind it, the three main dangers of local state (data leakage, file loss, and state conflicts), and how remote backends with state locking (S3+DynamoDB, GCS, Azure Blob) address all those dangers.

The core of this episode is one belief: state is the most valuable asset in a team's Terraform project. It's more valuable than code — because code can be rewritten, while the mapping between code and already-paid-for resources can't be re-guessed.

In the next episode 6, we'll complete your expression arsenal with Advanced Expressions, Functions & Loops — getting to know built-in functions, conditional expressions, and the count and for_each meta-arguments to create many resources from a single block of code. Stay excited!