Progressive Delivery on Kubernetes with Argo Rollouts or Flagger

Why a rolling update is not enough

Kubernetes rolling updates replace pods gradually and stop if new pods never become ready. That protects you from a container that crashes on start. It does nothing for a release that starts fine and then returns errors on one endpoint, doubles a query’s latency, or breaks checkout for customers on one payment provider. By the time someone notices, every pod is already on the new version.

Progressive delivery adds the missing feedback loop. A small slice of real traffic meets the new version first, metrics decide whether to continue, and a failing check reverses the change without waiting for a human. This page explains how we build that as part of our GitOps, Helm, ArgoCD and Flux service, and where it is not worth the effort. If you are not yet deploying through git, start with the migration to GitOps.

What good looks like

• Every production release of a critical service goes through a staged rollout, and nobody has to remember to do it.
• The signals that stop a release are written down, versioned in git, and reviewed like code.
• A failed canary reverts on its own, and the team is told why.
• Promoting or aborting by hand is a single command that every on-call engineer knows.
• The same manifests work in staging, so the process is rehearsed constantly rather than only during incidents.

Canary, blue-green or neither

For stateful workloads and anything that changes a database schema, traffic shifting only helps if the schema change is backward compatible. We insist on expand and contract migrations: add the new column first, deploy code that handles both, then remove the old one later. Without that, a clean rollback of the code can still leave the data in a state the old version cannot read.

Argo Rollouts or Flagger

The two tools solve the same problem differently.

Argo Rollouts introduces a Rollout resource that replaces the Deployment. You define steps such as set weight, pause and analyse. It pairs naturally with ArgoCD, and its kubectl plugin and dashboard make a rollout easy to inspect.

Flagger leaves your Deployment in place and creates a primary and a canary copy around it, driven by a Canary resource. It sits well with Flux and with teams who would rather not rewrite workload manifests.

We choose on three questions: which GitOps controller you run, which ingress or mesh carries traffic, and whether converting Deployments to Rollouts is acceptable. Mixing the combination you do not already know usually costs more than the tool difference is worth.

How it works in practice

A canary with metric gates

This Argo Rollouts excerpt sends 10 percent of traffic to the new version, checks error rate, then proceeds in steps. The analysis template queries Prometheus:

apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: payments-api
spec:
  replicas: 6
  strategy:
    canary:
      canaryService: payments-api-canary
      stableService: payments-api-stable
      trafficRouting:
        nginx:
          stableIngress: payments-api
      steps:
        - setWeight: 10
        - pause: { duration: 5m }
        - analysis:
            templates:
              - templateName: error-rate
        - setWeight: 50
        - pause: { duration: 10m }
---
apiVersion: argoproj.io/v1alpha1
kind: AnalysisTemplate
metadata:
  name: error-rate
spec:
  metrics:
    - name: http-5xx-ratio
      interval: 1m
      failureLimit: 2
      successCondition: result[0] < 0.01
      provider:
        prometheus:
          address: http://prometheus.monitoring:9090
          query: |
            sum(rate(http_requests_total{service="payments-api-canary",code=~"5.."}[2m]))
            / sum(rate(http_requests_total{service="payments-api-canary"}[2m]))

The threshold of one percent is an example. Your own number comes from the service’s SLO and its normal behaviour, not from a template.

Blue-green with a manual gate

For blue-green, the new ReplicaSet comes up behind a preview service. Automated checks run against it, then a person or a pipeline step promotes it. This suits releases where you want a human to look at the preview before any customer does. Flagger supports the same pattern with its blue-green deployment strategy.

Getting the metrics right

Rollback automation fails quietly when the query is wrong. A query that returns no data may be treated as a pass or a failure depending on settings, and low traffic can make percentages swing wildly. We handle these on purpose:

• Decide whether “no data” blocks or passes, and test both.
• Use a minimum request count before trusting a ratio.
• Compare the canary against the stable version, not only against a fixed number, where traffic is spiky.
• Keep the queries next to your Prometheus and Grafana rules, so the same definition of “healthy” is used in alerts and in rollouts.

Common failure modes

• Rollouts that never finish because a pause step waits for a human who is not looking. We add notifications and time limits.
• Analysis on the wrong service after a rename, so everything passes. Rehearsal with a broken build catches this.
• ArgoCD fighting the controller over replica counts or weights. Ignore rules are set deliberately, as described under drift detection.
• Canary hitting a shared cache or queue and polluting stable traffic. Some services need a different strategy.
• Schema changes shipped with code and no safe way back.

When not to do this

If a service has little traffic, a canary receives too few requests to say anything statistically useful, and a blue-green with a smoke test is better. If you have no metrics yet, fix that first with monitoring. Batch jobs and internal tools rarely justify the extra moving parts.

Delivery and handover

We deliver the controller installation, rollout manifests, analysis templates, dashboards and a runbook covering abort, promote, retry and how to read a failed analysis. Notifications tell the team when a rollout pauses or reverts. After the first few real releases we review thresholds with you, because the initial numbers are always a best guess.

How to tell it is working

Failed releases are caught at a small percentage of traffic, rollbacks happen without a page, and engineers stop treating production deploys as an event. Review the number of aborted rollouts and what they caught. If you want to explore a pilot on one service, book a consultation.

References

• Canary, Argo Rollouts documentation
• Deployment Strategies, Flagger documentation