Kubernetes Autoscaling Patterns — A Field Guide

Kubernetes gives you four different autoscalers that all use the word "scale" and mean different things by it: the Horizontal Pod Autoscaler adds and removes pod replicas, the Vertical Pod Autoscaler resizes the resource requests of existing pods, the Cluster Autoscaler adds and removes nodes to fit pods that can't be scheduled, and Karpenter does the node job differently by provisioning capacity directly instead of managing fixed node groups. Most of the outages I've seen with autoscaling come not from any one of these misbehaving on its own, but from two of them fighting over the same signal without anyone having decided who wins.

HPA mechanics, and why it thrashes

The Horizontal Pod Autoscaler polls metrics — CPU and memory via metrics-server by default, or anything exposed through the custom or external metrics APIs (queue depth, request latency, a Prometheus query via the Prometheus Adapter) — and adjusts replicas on a Deployment or StatefulSet to keep the observed value near a target. The naive mental model is a thermostat, but a thermostat with a slow, noisy sensor: metrics-server refreshes on an interval, pods take time to become ready, and a burst of traffic can trigger a scale-up before the metric has settled, followed by a scale-down once the new pods start reporting lower per-pod load, followed by another scale-up once the reduced pod count pushes load back up. That oscillation is thrashing, and it's the single most common HPA complaint.

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: checkout-api
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: checkout-api
  minReplicas: 3
  maxReplicas: 20
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 65
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
        - type: Pods
          value: 2
          periodSeconds: 60
    scaleUp:
      stabilizationWindowSeconds: 30

The fix is the behavior block: a scale-down stabilization window makes the HPA look at the highest recommendation over the trailing window rather than reacting to the latest sample, so a brief dip doesn't immediately shed capacity you'll need again in two minutes. Most thrashing complaints I've debugged trace back to an HPA running on defaults with no behavior block set at all.

VPA modes, and why Recreate evicts pods

The Vertical Pod Autoscaler solves a different problem: getting the CPU/memory requests and limits right for a workload whose usage doesn't match what a human guessed at deploy time. It runs in one of four modes — Off (recommendations only, nothing applied), Initial (sets requests only at pod creation), Recreate (evicts and recreates running pods to apply new resource values), and Auto (currently behaves like Recreate). The gotcha is in the name: Recreate and Auto modes don't patch a running pod's resources in place — Kubernetes doesn't support that for most resource types — they evict the pod and let its controller recreate it with updated values. On a workload with few replicas or a short terminationGracePeriodSeconds, that eviction is a real disruption, not a free lunch.

Cluster Autoscaler: easy to scale up, hard to scale down

Cluster Autoscaler watches for pods that fail to schedule because no node has enough free capacity, and adds nodes from a configured node group to fit them — that half is fairly mechanical. Scale-down is where it gets interesting: Cluster Autoscaler will only remove a node if every pod on it can be safely evicted and rescheduled elsewhere, and there's a long list of things that block that. A PodDisruptionBudget that would be violated by evicting a pod stops the drain outright. A pod using local storage (emptyDir with data the app cares about, or a hostPath volume) is treated as non-evictable by default. A pod without a controller (a bare Pod, not part of a Deployment/StatefulSet/Job) also blocks it, since Cluster Autoscaler can't be sure it'll come back. The practical result: clusters that scale up fine under load but never scale back down, quietly costing money on nodes nothing is actively blocking removal of — except one PDB somewhere set to minAvailable: 100%.

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: checkout-api-pdb
spec:
  maxUnavailable: 1
  selector:
    matchLabels:
      app: checkout-api

Karpenter: skipping node groups entirely

Karpenter takes a different approach to the same problem Cluster Autoscaler solves. Instead of scaling a predefined, fixed-shape node group (all the same instance type, managed as a unit), Karpenter watches for unschedulable pods and provisions exactly the node shape those pods need, directly from the cloud provider's API, at the moment it's needed. That means better bin-packing — it can pick from a wide range of instance types and sizes to match what's actually pending, rather than over-provisioning a uniform node group to cover the worst case — and it collapses the node-group-per-workload-shape sprawl that Cluster Autoscaler setups tend to accumulate over time. The trade-off is that Karpenter's consolidation behavior (actively repacking and terminating underutilized nodes, not just declining to remove them) is more aggressive by default, so workloads that are sensitive to being rescheduled need the same PDB and graceful-shutdown discipline as they would under Cluster Autoscaler, just triggered more often.

Wrapping up

The autoscalers aren't competitors, they operate at different layers — HPA and VPA at the pod level, Cluster Autoscaler and Karpenter at the node level — but every incident I've traced back to autoscaling came from a layer being configured as if it were alone: HPA with no stabilization window fighting its own noisy metric, VPA in Auto mode resizing the exact metric HPA is scaling on, or a scale-down blocked for weeks by a PDB nobody remembered setting. Get the stabilization windows, the mode choices, and the disruption budgets right once, and the rest of it really does run itself.