Capacity Planning and Load Testing Before Traffic Finds the Limits

Why launches fail on the day

Most capacity incidents follow the same script. A campaign or release lands, traffic climbs, and something unexpected saturates first: the database connection pool, a single-threaded queue consumer, a rate-limited third-party API, or a node group that cannot scale quickly enough. CPU on the web tier was fine. Nobody tested the path that actually broke.

Capacity planning and load testing exist to find that first weak point while it is cheap to fix. It is part of our Managed DevOps service, and it links closely to cost reviews, since the same data tells you where you are overbuilt as well as underbuilt.

What good looks like

• A written statement of the demand you must handle: peak requests per second, concurrent users, data growth, and the events that cause spikes.
• Defined success criteria such as p95 latency and error rate under that load.
• Load tests that reproduce real user journeys, not a single endpoint hammered.
• A known first bottleneck and a known second one.
• Autoscaling configured from test evidence, not defaults.
• Headroom that survives the loss of an availability zone.

How it works in practice

Start with a traffic model

Raw volume is the least interesting number. What matters is the mix: reads versus writes, cacheable versus uncached, cheap versus expensive endpoints, anonymous versus authenticated. We derive it from access logs and analytics, then add the growth assumption your business plans for. Arrival rate matters too. Real traffic is open: users arrive whether or not the server is keeping up. Many naive tests are closed-loop, where each virtual user waits for a response, which hides overload because the test slows down with the system.

A representative k6 test

An arrival-rate scenario keeps pressure constant regardless of response time:

import http from 'k6/http';
import { check, sleep } from 'k6';

export const options = {
  scenarios: {
    checkout_peak: {
      executor: 'ramping-arrival-rate',
      startRate: 20,
      timeUnit: '1s',
      preAllocatedVUs: 200,
      maxVUs: 1000,
      stages: [
        { target: 150, duration: '5m' },
        { target: 150, duration: '15m' },
        { target: 0, duration: '2m' },
      ],
    },
  },
  thresholds: {
    http_req_failed: ['rate<0.01'],
    http_req_duration: ['p(95)<800'],
  },
};

export default function () {
  const res = http.get(`${__ENV.BASE_URL}/api/products`);
  check(res, { 'status is 200': (r) => r.status === 200 });
  sleep(1);
}

The numbers above are examples; real targets come from your own goals. Thresholds make the test pass or fail automatically, so it can run in a pipeline.

Watch the system, not just the tester

The test client reports what users would feel. Prometheus and Grafana show why. During every run we watch saturation at each tier: CPU throttling, memory, connection pool wait time, database locks and replication lag, queue depth, and node scaling events. Distributed traces help find the slow hop. Our monitoring page describes the signals we already collect.

Find and fix the bottleneck

Typical findings, roughly in order of frequency:

1. Database: missing indexes, N+1 queries, undersized connection pools, or a single primary handling reads.
2. Application: synchronous calls that should be queued, blocking work on request threads.
3. Autoscaling: scaling that triggers too late, or pods that take minutes to become ready.
4. Cluster: no node capacity for new pods, or quotas hit.
5. Third parties: rate limits and slow responses that cascade.

We fix one thing, rerun the same test, and compare. Changing five things at once leaves you unsure what helped.

Turning results into settings

Load tests are worth little if they stay in a slide deck. We convert them to configuration:

• Pod requests and limits based on measured usage at target load.
• Horizontal pod autoscaler thresholds and minimum replicas so scaling begins before saturation. Event-driven scaling with KEDA suits queue-based workloads.
• The vertical pod autoscaler in recommendation mode to keep requests honest over time.
• Node capacity and autoscaler settings, including how fast new nodes arrive. See cluster operations.
• Alerts on saturation trends, so the next limit is visible before it is hit.

Decisions and trade-offs

Test environment fidelity. A smaller staging environment can still reveal bottlenecks, but results do not scale linearly. We state what a result does and does not prove.

Pre-scaling versus autoscaling. A flash sale may spike faster than nodes can start. Pre-scaling before a known event is cheap insurance. Autoscaling is better for organic variation.

Over-provisioning. Running at peak capacity all year is safe and costly. The goal is enough headroom, not maximum headroom.

When not to do this. An early product with a few hundred users has more urgent problems than a formal capacity study. A simple smoke-level load test before a launch is usually enough. The full process pays off when downtime has a clear cost.

A worked example of reading results

Suppose a test ramps to the target rate and p95 latency climbs after twelve minutes while CPU on the application pods sits at a modest level. Adding replicas does not help. Looking one layer down, the database shows connections waiting on a pool capped at a low number, and a single slow query holding locks. The fix order is then clear: add the missing index, raise the pool carefully (a larger pool can overload the database), and retest. Only after that does it make sense to tune autoscaling.

This pattern repeats often enough that we treat pods and CPU as the last place to look, not the first. A useful habit is to write down a hypothesis before each run (“we expect the database to saturate first”) and compare it with what happens. Wrong guesses are where the learning is.

Common failure modes

• Testing with an empty database. Queries behave differently on realistic data volumes.
• Hitting real third parties. Use sandboxes or mocks, or you pay and risk being blocked.
• Ignoring cold starts. Fresh pods and cold caches behave differently from warm ones.
• One-off tests. Performance regresses release by release. We keep a lightweight test in CI for key paths.
• Forgetting the downstream. Your service survives, then the shared database or queue does not.

Delivery and handover

You get the test scripts in your repository, a short report of limits and bottlenecks, updated autoscaling configuration, and a capacity forecast reviewed with you in the monthly report. Before major releases we pair this with an architecture review.

How to tell it is working

Look for a known maximum sustainable load, a stable p95 latency at expected peak, autoscaling that activates before alerts fire, and launch days that are uneventful. To plan for an upcoming event, book a consultation.

References

• Load test types, Grafana k6 documentation
• Thresholds, Grafana k6 documentation
• Automated performance testing, Grafana k6 documentation