Karpenter or Cluster Autoscaler: Scaling EKS Capacity Efficiently

Scaling is where EKS money is won or lost

Compute is usually the largest line on an EKS bill, and most of the waste comes from nodes that are bigger or more numerous than the workload needs. Autoscaling should fix that, but a poorly configured autoscaler can instead cause pending pods during traffic spikes, or evict the wrong things at the wrong moment.

This page describes how we select and configure node autoscaling as part of AWS EKS consulting. The financial side is covered in cost analysis and optimisation, and the broader tactics in our guide to EKS cost optimisation.

What good looks like

• Pods that need capacity get it quickly, and capacity that is idle goes away.
• Instance types are chosen from a sensible set, not hard-coded to one family.
• Spot is used where interruption is acceptable and not elsewhere.
• Scale-down never breaks availability because disruption budgets are respected.
• You can see scaling decisions in dashboards and logs.

Karpenter or Cluster Autoscaler

We recommend Karpenter for most new production clusters because it reduces the tuning that fixed node groups need. We recommend Cluster Autoscaler when the team already runs it well and the workload is predictable. We do not recommend switching purely for novelty.

A representative Karpenter configuration

Karpenter uses a NodePool to describe what it may launch and an EC2NodeClass to describe how. The example below allows a Spot and on-demand mix across several families, caps total capacity and enables consolidation. API versions change between Karpenter releases, so confirm against the version you install.

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: general
spec:
  template:
    spec:
      nodeClassRef:
        group: karpenter.k8s.aws
        kind: EC2NodeClass
        name: default
      requirements:
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["spot", "on-demand"]
        - key: karpenter.k8s.aws/instance-category
          operator: In
          values: ["c", "m", "r"]
        - key: kubernetes.io/arch
          operator: In
          values: ["amd64"]
      expireAfter: 720h
  limits:
    cpu: "400"
  disruption:
    consolidationPolicy: WhenEmptyOrUnderutilized
    consolidateAfter: 1m
    budgets:
      - nodes: "10%"

The limits block is a cost safety net: it stops a runaway deployment from launching unlimited nodes. The budgets entry controls how many nodes may be disrupted at once.

Spot without drama

Spot saves money but can be reclaimed with short notice. Our rules of thumb:

• Allow many instance types and sizes so capacity pools are deep.
• Run at least two replicas, spread across zones, for any service on Spot.
• Keep databases, singletons and long-running non-resumable jobs on on-demand nodes using separate NodePools and taints.
• Handle interruption gracefully: Karpenter reacts to interruption events via an SQS queue, and applications need sensible termination grace periods.

Requests first, autoscaler second

An autoscaler schedules on pod requests, not actual usage. If every pod asks for far more than it uses, the cluster fills with reserved but idle capacity, and no scaler can fix that. Before tuning, we compare requests with measured usage and adjust. Pair this with the Horizontal Pod Autoscaler for replica scaling, and consider event-driven scaling for queue workers. Background on resource requests and limits explains why this matters.

Separating workload classes

One NodePool rarely suits everything. We usually define a small number of classes and let scheduling rules steer pods to them. A general class carries stateless services on a Spot and on-demand mix. A protected class on on-demand capacity carries databases, singletons and anything that cannot tolerate interruption, and is tainted so only workloads that tolerate the taint land there. A specialised class carries GPU or high-memory pods, with tight limits so a mistaken deployment cannot run up a bill.

# pod spec fragment for a workload that must stay on protected capacity
tolerations:
  - key: "workload-class"
    operator: "Equal"
    value: "protected"
    effect: "NoSchedule"
nodeSelector:
  workload-class: protected

Keeping the number of classes small matters. Each extra class is another set of limits and disruption rules to understand, and capacity in many small pools packs less efficiently than capacity in a few shared ones.

A worked example: a slow scale-up

A common complaint is that pods wait too long during a traffic burst. We trace the delay in stages: how long until Karpenter notices the pending pod, how long until the instance launches, how long until the node joins and becomes ready, and how long until the image pulls. The slow stage differs between clusters. If image pulls dominate, smaller images or a cache help more than any autoscaler setting. If launches fail, the cause is often an instance quota or an unavailable capacity pool, which broader instance choices solve.

Failure modes we plan for

• Consolidation churn: aggressive settings keep replacing nodes. We tune consolidateAfter and budgets, and protect sensitive pods with annotations or PodDisruptionBudgets.
• Pending pods during bursts: caused by tight limits, zone constraints or instance quotas. We review EC2 service quotas ahead of launch.
• IP exhaustion: many small nodes consume subnet IPs. This ties back to architecture and subnet sizing.
• Karpenter as a single point: it needs to run on capacity it does not manage itself. We place it on a small stable node group.
• Slow image pulls: new nodes feel slow when images are large. We discuss image size and caching.

How it is delivered

We deploy the autoscaler with Terraform and Helm, configured through code, with test scenarios documented. Your team receives guidance on adding a NodePool, reading Karpenter logs and changing limits. If you run on Terraform modules, the IAM roles, queue and instance profile are part of the same module set.

How to measure it

Watch pending pod duration, node utilisation (requested versus allocatable), the share of capacity on Spot, interruption counts and provisioning time from pending to ready. Combine these with a cost view in Prometheus and Grafana. If utilisation stays low after tuning, the usual culprit is oversized requests rather than the scaler. For help diagnosing yours, book a consultation.

References

• NodePools, Karpenter documentation
• Disruption, Karpenter documentation
• Karpenter, Amazon EKS Best Practices Guide
• Cluster Autoscaler, Amazon EKS Best Practices Guide