The bill nobody was watching
Cloud spend fails quietly. A load test left running, a NAT gateway carrying terabytes it should not, a leaked key mining cryptocurrency: each looks fine in the console and horrifying on the invoice a month later. The pattern we see most often in console-built estates is that no one set up alerts at all, because the first month’s bill was small.
Budgets and alerts are cheap to set up and are among the first things we do in a cloud setup engagement. They also work with the account design in multi-account structure: account boundaries make spend attributable without any tagging at all, and tags add a finer view inside each account.
What good looks like
- A budget exists for the organisation as a whole and for every account.
- Forecast alerts fire before the month ends, with enough time to react.
- Key services that tend to surprise (data transfer, NAT, logs, GPUs) have their own budgets.
- Cost allocation tags are consistent, activated and enforced.
- Anomaly detection watches for sudden changes.
- Alerts reach someone who can act, and that person knows what normal looks like.
- Spend is reviewed on a regular schedule.
How it works in practice
Billing foundations
In the management account, we enable Cost Explorer, turn on the preferences that allow member accounts to view their own costs if you want that, and set the alternate billing contact so invoices and notices reach finance. The root account hardening work includes an early budget, because a cost alarm is a useful tripwire for account compromise too.
Budgets as code
A monthly budget with actual and forecast notifications, in Terraform:
resource "aws_budgets_budget" "prod_monthly" {
name = "prod-monthly"
budget_type = "COST"
limit_amount = var.prod_monthly_limit
limit_unit = "USD"
time_unit = "MONTHLY"
cost_filter {
name = "LinkedAccount"
values = [var.prod_account_id]
}
notification {
comparison_operator = "GREATER_THAN"
threshold = 80
threshold_type = "PERCENTAGE"
notification_type = "FORECASTED"
subscriber_email_addresses = var.finance_and_owner_emails
}
notification {
comparison_operator = "GREATER_THAN"
threshold = 100
threshold_type = "PERCENTAGE"
notification_type = "ACTUAL"
subscriber_email_addresses = var.finance_and_owner_emails
}
}
The limit is a variable on purpose: we set it from what the account really costs, then tighten it as the estate stabilises. Alerting at 80 percent forecast and 100 percent actual is a start, and more granular steps can follow.
Tags that actually work
A tag is only useful if it is consistent. We agree a short list, usually team, environment, product and owner, and apply four rules. They are activated as cost allocation tags in the billing console, which takes effect from activation onward and does not backfill history. They are set by default at the Terraform provider level so that every resource carries them. They are validated by tag policies and Config rules. Untagged resources are reported to the owning team rather than silently tolerated.
provider "aws" {
default_tags {
tags = {
team = "platform"
environment = "production"
managed_by = "terraform"
}
}
}
This is one of the quiet benefits of the Terraform approach: tagging stops depending on memory.
Anomaly detection
Cost Anomaly Detection creates monitors for services, linked accounts or tag values, and sends alerts when spend deviates from learned patterns. It catches the case where you are under budget overall but one service has jumped. We typically create an organisation-wide service monitor and one per production account, with alert thresholds expressed as an absolute amount so that trivial changes stay silent.
Beyond alerts
Alerts tell you something is wrong. Finding the cause uses Cost Explorer grouped by service, account, tag and usage type, and for detail the Cost and Usage Report queried with Athena. Savings then come from reviewing commitments, rightsizing and architecture choices. For container platforms, see our EKS cost optimisation guide. If you want that done regularly, our Managed DevOps work includes cost reviews.
Decisions and trade-offs
Budget per account or per tag? Per account is simplest and robust because it needs no tagging discipline. Add tag budgets where several teams share an account.
How tight? Very tight budgets cause alert fatigue; very loose ones miss problems. We start generous, then tighten after two or three months of data.
Automatic enforcement? Budget actions that stop resources suit sandboxes. In production, we alert humans.
Pre-deployment estimates. Tools such as Infracost show cost changes in pull requests, which prevents some surprises before they exist.
When not to over-engineer. An early-stage product with a small, predictable bill needs one budget, one anomaly monitor and a monthly glance, not a finance operations function.
What a first review usually finds
When we look at spend for the first time in a console-built estate, the same few themes recur. Idle resources are common: unattached volumes, old snapshots, load balancers with no targets and development databases running around the clock. Data transfer is the next theme, particularly traffic through NAT gateways and between availability zones that nobody planned. Logging is a third: verbose application logs or flow logs sent to a premium tier at a volume no one reads. Finally, there is often an absence of commitment planning, where steady production workloads pay on-demand prices because nobody owns the decision. Budgets and anomaly alerts do not fix these, but they make them visible, which is the point at which fixing them becomes a normal engineering task rather than a finance surprise.
Common failure modes
- Alerts to a departed person’s inbox. Use distribution lists and chat channels.
- Tags activated late. Months of history remain unattributed.
- Forgetting data transfer and support. They hide in “other” categories.
- Credits masking reality. Promotional credits hide the true run rate until they expire. We budget on gross usage as well.
- No owner. An alert everyone receives is one nobody answers.
Delivery and handover
We deliver budgets, monitors and tag rules as code, a one-page description of who owns each alert and what to do when it fires, and a template for the monthly review. For multi-account setups, the same module creates a budget automatically for each new account.
How to tell it is working
- A deliberate test (a temporary expensive resource in a sandbox) triggers an alert quickly.
- Every account shows a named owner and a budget.
- Tag coverage on cost reports is high and rising.
- Nobody learns about a cost spike from the invoice.
To go through your current spend and alert setup, book a consultation.