Structuring Terraform for Dev, Staging and Production Environments

Why environment layout is a safety decision

Every team eventually wants a place to test infrastructure changes before production. The way Terraform is laid out decides whether that test environment is trustworthy or a trap.

Three problems appear again and again. Staging is a hand-edited copy that no longer resembles production. An engineer runs terraform apply with the wrong workspace selected. Or a single giant configuration means that changing a dev setting forces a plan across production resources. A good Terraform multiple environments design removes all three. This page is part of our Terraform and infrastructure as code service.

What good looks like

• Dev, staging and production use the same modules, differing only in input values.
• Each environment has its own state and its own credentials.
• It is hard to apply to the wrong place: the directory, backend and role all agree.
• Changes promote in order, and the production plan is always reviewed by a human.
• Environment-specific safety, such as deletion protection, is on where it matters.

Workspaces or separate state

This is the question we are asked most. Here is how they compare:

Our default for permanent environments is separate directories. For ephemeral per-pull-request environments, workspaces or generated state keys work well, as long as something cleans them up afterwards.

How it works in practice

A repository layout that scales

infrastructure/
  modules/
    network/
    cluster/
    database/
  environments/
    dev/
      main.tf
      backend.tf
      terraform.tfvars
    staging/
      main.tf
      backend.tf
      terraform.tfvars
    prod/
      main.tf
      backend.tf
      terraform.tfvars

Each main.tf is short: a handful of module calls pinned to a version, fed by variables. Production and staging should differ in terraform.tfvars and not in main.tf. If you find yourself editing structure per environment, the module probably needs a new input.

Separate state and separate identity

The backend key identifies the environment, and the role used by the pipeline can only read and write that environment’s state and resources:

terraform {
  backend "s3" {
    bucket       = "example-tfstate-eu-west-1"
    key          = "prod/platform/terraform.tfstate"
    region       = "eu-west-1"
    encrypt      = true
    use_lockfile = true
  }
}

The strongest version of this is a separate AWS account or Azure subscription per environment, organised under AWS Organizations or Azure management groups. Then even a compromised dev pipeline cannot reach production. Details of the backend are in our guide to remote state on S3 and Azure Blob.

Promotion

Changes flow through the same pipeline each time. A pull request shows the dev plan, merge applies it, and the same module version is then proposed for staging and finally production. We describe the automation side in Atlantis and Terraform Cloud pipelines. Production applies require an approval, and they happen from the pipeline rather than a laptop.

Sharing values without sharing state

Environments often need facts from each other’s foundations, such as a VPC ID or a cluster endpoint. Reading another stack’s raw state couples the two tightly, and it hands the reader access to everything in that state. A lighter option is to look the value up from the cloud itself:

data "aws_vpc" "main" {
  tags = {
    Name = "prod-main"
  }
}

The lookup depends on a stable tag convention, which is a fair price for keeping stacks independent. Where a lookup is awkward, a small published output, such as a parameter store entry, does the same job.

Decisions and trade-offs

Should staging match production exactly? Match structure, not size. Staging can use smaller instances and a single replica, but it should have the same network layout, the same modules and the same policies. The point is to catch configuration failures, not to load test. For capacity questions use a dedicated test, not staging.

How many environments? Dev, staging and production is a sound default. Add more only when a real need appears, such as a compliance sandbox. Every extra environment is more to keep in sync and more to pay for.

DRY versus readable. Terragrunt or heavy templating reduces repetition but makes the final configuration harder to read. We tolerate a little duplication in main.tf for clarity when there are only a few environments.

When not to separate accounts. For a two-person startup with no customer data, one account is a reasonable start. We would still lay out the code so moving later is a backend change and not a rewrite.

Failure modes we avoid

• Environment drift through manual hotfixes. Detecting this is covered in drift detection and remediation.
• Shared remote state reads that create tight coupling. We prefer data sources or published outputs over reaching into another team’s state.
• Variables files with secrets. Secrets come from a secrets manager at runtime, not from committed tfvars.
• Untested production only features. If a setting only exists in production, nobody tests it. We keep exceptions small and explicit.

Adding the next environment

A good layout proves itself when someone adds a fourth environment, for example a temporary one for a customer demo. The steps should be short: copy the thin environment folder, set a new backend key, fill in the variable values, create the role the pipeline will assume, and open a pull request. If any step needs a meeting, that step is missing from the documentation or from the module inputs, and we fix it then.

Delivery and handover

You receive the repository structure, pipeline wiring per environment, and a short guide on adding a new environment. The first additional environment is built together with your engineers so the process is understood, not just delivered.

How to tell it is working

Spinning up a new environment takes hours, not weeks. The diff between staging and production configuration is a short, explainable list of values. Nobody has applied to the wrong environment in months. If you would like us to review your current layout, talk to us.

References

• State: Workspaces, Terraform, HashiCorp Developer
• Standard Module Structure, Terraform, HashiCorp Developer