Learn Terraform - Workspaces vs Directory-Based Multi-Environment

#Introduction

After discussing Creating & Managing Reusable Modules in episode 9 — how to package infrastructure into DRY, reusable components — in this episode we face an equally important follow-up question: that same module, which environments is it going to be used for, and how do we manage it safely?

In the real world, there's never an application that only runs in one environment. At minimum there's dev for daily experimentation, staging for production-like testing, and prod for real traffic. All three need infrastructure with the same structure — VPC, subnets, instances, databases — but with different values (instance size, number of subnets, CIDR, tags). More importantly: all three must be mutually isolated. If an error happens in dev, prod shouldn't be affected. If prod state is locked during a release, dev must still be usable.

The question is, how do you manage three (or more) of these environments with Terraform? There are two major approaches we'll dissect thoroughly in this episode: Terraform Workspaces and Directory-Based Structure. Both are often misunderstood as the same thing, even though their philosophy, advantages, and limitations are very different — and a wrong choice here will be felt painfully months later.

#Main Discussion

#The Fundamental Challenge of Multi-Environment

Before choosing a strategy, let's first define what must be isolated. From field experience, there are four things that must be separated between environments:

1. State file — prod state must not mix with or be overwritten by dev state.
2. Cloud credentials — access to the production account must not be used for dev experiments.
3. Blast radius — an error in one environment must not cascade into another.
4. Change process — releases to prod usually require manual approval; dev doesn't.

The tighter the isolation, the safer — but the heavier the management overhead. Workspaces and directory-based are two points on this trade-off spectrum.

#Terraform Workspaces: Concept and Usage

A workspace is a separate state instance within one configuration and one shared backend. Think of it like several branches in one state repo: the same code, but different states.

If the backend used is S3, each workspace's state is stored with the env:/<workspace-name>/ prefix, for example env:/dev/terraform.tfstate and env:/prod/terraform.tfstate. In practice you start from the automatically-created default workspace, then create the others:

terraform workspace new dev

Created and switched to workspace "dev"!
 
You're now on a new, empty workspace. Workspace "dev" is the most recently
created. If you'd like to move existing resources, run "terraform workspace
select" and re-run your previous apply.

terraform workspace new staging
terraform workspace new prod

To switch between workspaces and see the current status:

terraform workspace list
terraform workspace select staging
terraform workspace show

  default
  dev
* prod
  staging

Notice the asterisk (*) — it indicates the currently active workspace. All subsequent plan/apply commands work against that active workspace's state.

#Using terraform.workspace in the Configuration

To differentiate values between environments, Terraform provides the built-in variable terraform.workspace, which contains the active workspace name. The most common pattern is creating a name_prefix and environment tags:

locals {
  name_prefix = "${var.project_name}-${terraform.workspace}"
 
  common_tags = {
    Environment = terraform.workspace
    Project     = var.project_name
    ManagedBy   = "terraform"
  }
}
 
module "vpc" {
  source = "./modules/vpc"
 
  name = local.name_prefix
  cidr = var.vpc_cidrs[terraform.workspace]
 
  tags = local.common_tags
}

With this pattern, terraform workspace select dev then terraform apply will produce a VPC named myapp-dev, while in prod it produces myapp-prod. Per-environment values like vpc_cidrs can be stored in a map indexed by the workspace name.

#The Often-Underestimated Limitations of Workspaces

Workspaces look practical — one code, one folder, several states. Unfortunately, this practicality comes at a high price at enterprise scale:

1. State isn't isolated. All workspaces live in the same backend (the same S3 bucket, the same DynamoDB lock). This means one set of credentials can touch every environment.
2. No credential isolation. Because it's one configuration and one backend, it's hard to force prod to only be accessible by authorized CI. Anyone who can apply in dev can technically workspace select prod then apply as well.
3. Risk of accidental switch. The terraform apply command doesn't re-ask which workspace is active. Identical configuration for all workspaces makes mistakes invisible until everything is already applied. The state lock also only locks one workspace — it doesn't prevent two people from changing dev and prod at the same time.
4. Same blast radius. A code error that makes Terraform destroy resources will hit whichever environment is being applied.
5. terraform.workspace values baked into state. If you ever want to rename a workspace or change strategy, moved/rename becomes painful because the prefix in resources changes too.

#Directory-Based Structure: The Enterprise Standard

The second approach, and the industry-standard recommendation, is the directory-based structure: each environment is a physically separate Terraform configuration — its own directory, its own state backend, its own credentials, and its own execution flow. All environments share the same modules in the modules/ directory, so the code stays DRY, but the isolation is complete.

terraform-infra/
├── modules/
│   ├── vpc/                 # Shared module, used by all environments
│   └── ec2/
├── environments/
│   ├── dev/
│   │   ├── backend.tf       # Dev-specific S3 backend
│   │   ├── main.tf
│   │   ├── variables.tf
│   │   ├── locals.tf
│   │   └── terraform.tfvars # Dev-specific values
│   ├── staging/
│   │   ├── backend.tf
│   │   ├── main.tf
│   │   ├── variables.tf
│   │   ├── locals.tf
│   │   └── terraform.tfvars
│   └── prod/
│       ├── backend.tf
│       ├── main.tf
│       ├── variables.tf
│       ├── locals.tf
│       └── terraform.tfvars

Each environment has its own state backend. Notice the different key in each backend.tf file:

terraform {
  backend "s3" {
    bucket         = "company-terraform-state"
    key            = "prod/terraform.tfstate"
    region         = "ap-southeast-1"
    encrypt        = true
    dynamodb_table = "terraform-locks"
  }
}

terraform {
  backend "s3" {
    bucket         = "company-terraform-state"
    key            = "dev/terraform.tfstate"
    region         = "ap-southeast-1"
    encrypt        = true
    dynamodb_table = "terraform-locks"
  }
}

Environment-specific values are stored in terraform.tfvars, then mapped into tidy locals for the whole configuration to use:

environment       = "dev"
vpc_cidr          = "10.10.0.0/16"
instance_type     = "t3.micro"
enable_nat_gateway = false

locals {
  name_prefix = "${var.project_name}-${local.environment}"
 
  common_tags = {
    Environment = local.environment
    Project     = var.project_name
    ManagedBy   = "terraform"
  }
}
 
module "vpc" {
  source = "../../modules/vpc"
 
  name = local.name_prefix
  cidr = var.vpc_cidr
 
  tags = local.common_tags
}

Notice source = "../../modules/vpc" — the module is referenced from the environment directory's position. The core configuration (main.tf) in each environment becomes nearly identical because the differences are only variable values.

The execution flow is also totally separate — each environment is run from its own directory:

cd environments/dev && terraform init && terraform apply -auto-approve
cd environments/prod && terraform init && terraform apply -auto-approve

#Workflow: Workspace vs Directory

# One directory, state differentiated via env:/ prefix
terraform workspace new dev
terraform workspace select dev
terraform plan
terraform apply
 
terraform workspace select prod
terraform plan
terraform apply

# Different directories, different backends & credentials
cd environments/dev
terraform init
terraform apply -auto-approve
 
cd ../prod
terraform init
terraform apply -auto-approve

#Comparison: Workspaces vs Directory-Based

#Recommendation for You

A rule of thumb used by many teams:

• Use directory-based as the default for permanent, long-lived environments (dev, staging, prod). The initial investment is slightly larger, but the benefits of state, credential, and process isolation pay off many times over.
• Use workspaces for ephemeral things — for example per-pull-request previews, or experiment sandboxes discarded when done. Workspaces are nice because they're cheap to create and destroy.
• Combine both — for example a dev/ directory using workspaces to differentiate each developer's environment (dev/arman, dev/siti), while staging/ and prod/ remain a single default workspace. This hybrid pattern is very common and the most flexible.

Whatever you choose, make sure these three things always exist: a clear name_prefix/environment tag per environment, state that can be traced back to which environment it belongs to, and a plan process always checked before apply.

#Common Pitfalls

1. Applying in the wrong workspace

A dangerous habit: busy in dev, moving to another task, then unknowingly still in the prod workspace when running terraform destroy. Always check with terraform workspace show before destructive commands, or get used to running terraform plan first and reading its output.

2. Using terraform.workspace for things that aren't environment

Values like instance size, number of AZs, or configurations that "happen" to differ between envs should be regular variables, not terraform.workspace. Hanging logic on the workspace name makes code hard to test and difficult if you later move to directory-based.

3. Ignoring credential isolation

workspace select prod with the same credentials as dev means prod can be destroyed by anyone with dev access. This is the strongest reason to choose directory-based with separate accounts/roles.

4. The same backend.tf for all environments

If all environments use the same key (e.g. terraform.tfstate), their states will overwrite each other — worse than workspaces. Make sure the key (or bucket) is truly unique per environment.

5. Duplicating environment code without modules

Good directory-based setups still share modules/. If each environment instead has its own copies of resources (copy-paste between folders), you're just moving the duplication problem, not solving it.

6. Unknowingly mixing workspace state with non-workspace

When a backend switches to workspace mode (env:/ prefix), old state at the old key becomes "orphaned". Do the migration deliberately (e.g. via terraform state mv to the new prefix), don't just switch backends carelessly.

#Conclusion

In this episode 10 we compared two multi-environment management strategies in depth: Terraform Workspaces — separate states within one configuration and one backend, practical for ephemeral environments but limited in state, credential, and blast radius isolation; and Directory-Based Structure — each environment has its own directory, backend, and credentials while still sharing modules, which is the enterprise standard for permanent dev, staging, and prod.

Key takeaways to bring home:

• The isolation that must be guaranteed: state, credentials, blast radius, and approval process.
• Workspaces share one execution session — risky for permanent environments.
• Directory-based lets CI/CD separate flows, credentials, and approval per environment.
• Use consistent name_prefix and environment tags, whether through terraform.workspace or locals + terraform.tfvars.

Now you have modular code organized per environment. But having "correct" code isn't enough — how do you make sure its quality and security are maintained before those changes reach prod? How do you catch errors early?

In the next episode 11 we'll discuss Formatting, Linting & Automated Testing — terraform fmt and validate, static analysis with tflint, security scanning with Checkov and Trivy, up to automated testing of real infrastructure with Terratest. Stay excited!