Designing Reusable Terraform Modules for VPCs, Kubernetes, Databases and IAM

The cost of copy and paste infrastructure

The first Terraform code in most companies is a single directory of resource blocks. That is fine for a month. Then staging is created by copying production and changing a few names, and a third environment follows. Soon a security fix, such as turning on encryption for a database, has to be made in five places, and one of them gets missed.

Reusable Terraform modules fix this by turning a pattern into a single, versioned unit. This page explains how we design them as part of our Terraform and infrastructure as code service, and, just as importantly, where modules go wrong.

What good looks like

A module is worth keeping when it meets these criteria:

• It encodes a decision. The S3 module always blocks public access and enables encryption. Nobody has to remember.
• It has a small, typed interface. Variables carry types, descriptions and validation rules.
• It returns useful outputs. IDs, ARNs and names that other modules need, and nothing more.
• It composes. A service stack calls a network module, a cluster module and a database module side by side, rather than one module calling another five layers deep.
• It is tested and documented. Covered in our pages on module documentation and versioning.

The modules we usually build

For clusters, see our EKS consulting and Kubernetes consulting services, which build on the same modules.

How it works in practice

A tight module interface

Here is the shape of a well-behaved storage module call. The caller sets a name and a few options, and everything else is policy:

module "assets_bucket" {
  source  = "git::https://github.com/example-org/terraform-aws-bucket.git?ref=v1.4.0"

  name        = "acme-assets-prod"
  versioned   = true
  lifecycle_days_to_ia = 60

  tags = local.common_tags
}

Inside the module, variables are typed and validated, so mistakes fail at plan time instead of apply time:

variable "name" {
  type        = string
  description = "Globally unique bucket name."

  validation {
    condition     = can(regex("^[a-z0-9][a-z0-9.-]{1,61}[a-z0-9]$", var.name))
    error_message = "Bucket name must be 3 to 63 characters, lowercase letters, numbers, dots and hyphens."
  }
}

Wrap, do not fork, community modules

For a VPC or an EKS cluster, the open source modules are mature. Our default is a thin internal wrapper around a pinned community module. The wrapper sets your CIDR conventions, tags and logging, while upstream does the heavy lifting. When upstream releases a fix, you bump one version. Forking is the last resort, because you then own every future bug.

Adopting modules without rebuilding anything

Moving existing resources into a module changes their addresses in state, which would normally mean destroy and recreate. A moved block tells Terraform it is the same object:

moved {
  from = aws_s3_bucket.assets
  to   = module.assets_bucket.aws_s3_bucket.this
}

We confirm a clean plan after every move. This is the same technique we describe for importing existing resources.

A plan-level test

Native tests let a module prove its own promises without creating anything. This one checks that the bucket module refuses a badly formed name:

run "rejects_uppercase_name" {
  command = plan

  variables {
    name = "Not-Valid"
  }

  expect_failures = [var.name]
}

Tests like this are cheap to run on every pull request. Apply-level tests that build real resources cost money and time, so we keep them for the behaviours a plan cannot prove, such as a bucket policy actually denying public reads.

Decisions and trade-offs

Flexible versus opinionated. A module with forty optional inputs is just a worse version of the provider. Prefer a narrow module plus a documented escape hatch, such as an input for extra tags or an extra policy document.

Module depth. Deeply nested modules make errors hard to trace and force changes through multiple layers. We keep trees shallow.

Terraform versus Terragrunt. Terragrunt reduces boilerplate when you have many near-identical stacks. For fewer than a handful of environments it is usually extra machinery.

When not to build a module. A resource used exactly once does not need one. Premature modules freeze the wrong abstraction. We typically wait until a pattern appears a second or third time.

Failure modes we avoid

• Hidden defaults that surprise people, such as a module that quietly creates a NAT gateway. Cost-bearing choices are explicit inputs.
• Breaking changes without a version bump. Renamed inputs or changed defaults get a major version and a changelog.
• Count and for_each tangles. Indexing resources by list position causes recreation when an item is removed. We key by stable names.
• Provider configuration inside modules. Modules receive providers from the caller, which keeps them reusable across accounts and regions.

Who owns the modules afterwards

A module library needs an owner, or it decays. We recommend naming a small group, often the platform team, who review changes to modules, publish releases and decide when a request for a new input is a real need rather than one team’s preference. Without that, the interface grows by accretion until it has the forty inputs we warned against above. Consumers should be able to open an issue, see a decision and know which version carries the answer.

Delivery and handover

You receive the modules in your own repository, with examples, tests and a contribution guide. We pair with your engineers on the first changes so ownership transfers properly, and the work links naturally into multi-environment layouts.

How to tell it is working

Creating a new environment becomes a short, boring pull request. Security fixes land in one module and roll out everywhere through version bumps. Plans stay free of unexpected changes after a module upgrade, and engineers outside the platform team can use the modules without asking for help. If that is where you want to be, get in touch.

References

• Creating Modules, Terraform, HashiCorp Developer
• Standard Module Structure, Terraform, HashiCorp Developer
• Module Composition, Terraform, HashiCorp Developer
• Tests, Configuration Language, Terraform, HashiCorp Developer