Kubernetes Cost Optimisation — A Field Guide

Why overprovisioned CPU and memory requests, not limits, are the biggest silent driver of Kubernetes spend, and how right-sizing, bin-packing, spot capacity, and cleanup actually bring the bill down. Every cluster I've inherited had the same story: requests set once at launch by copy-pasting a number that felt safe, never revisited, multiplied across dozens of deployments and every replica. Nobody notices because nothing breaks — the cluster just quietly reserves capacity nobody uses, and the invoice reflects it.

Requests drive cost, limits drive throttling

This is the distinction most teams get backwards. A pod's resources.requests is what the scheduler reserves on a node and what your cloud bill is actually shaped around — it's the number that determines how many pods fit per node and how many nodes you need. resources.limits only matters once the pod is running: exceed the CPU limit and you get throttled, exceed the memory limit and you get OOMKilled. A pod can run for months without ever touching its limits while its oversized requests sit there reserving capacity nobody uses, 24 hours a day, whether traffic is at 2am-quiet or peak. Overprovisioned requests are the single biggest silent cost driver in most clusters, and they're invisible unless you go looking, because nothing alerts on "this pod reserved 4x what it needs."

Right-sizing with historical data or VPA

The safest way to right-size is to pull actual CPU and memory usage over a real window — at least a couple of weeks, ideally including a traffic peak — and set requests close to observed p90-p95 usage rather than a guessed ceiling. The Vertical Pod Autoscaler can do this analysis for you if you run it in recommendation-only mode (updateMode: "Off"): it watches usage and produces a suggested request/limit, but doesn't evict or restart anything, which makes it safe to run everywhere as a signal rather than an active controller. I treat VPA recommendations as a starting point to review, not something to wire up in Auto mode blindly — a workload with a genuine, occasional spike will get a recommendation based on its quiet periods if you don't account for the spike deliberately.

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: checkout-api-vpa
  namespace: payments
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: checkout-api
  updatePolicy:
    updateMode: "Off"      # recommendation only, no evictions
  resourcePolicy:
    containerPolicies:
      - containerName: '*'
        minAllowed:
          cpu: 50m
          memory: 64Mi
        maxAllowed:
          cpu: 2
          memory: 2Gi

Bin-packing: fewer, larger nodes beat many small ones

Every node has fixed overhead — the kubelet, the CNI, system daemonsets, and a reserved slice of CPU/memory that's never schedulable. Ten small nodes carry that overhead ten times; two large nodes carry it twice. Beyond the raw overhead, small nodes fragment capacity: a node with 500m CPU free is useless to a pod that requests 750m, even though the cluster as a whole has plenty of room. Consolidating onto fewer, larger nodes — combined with pod topology spread constraints or node affinity so you don't sacrifice availability by putting all replicas on one physical node — typically recovers a meaningful chunk of "wasted" capacity that bin-packing math alone was losing to fragmentation. Cluster Autoscaler and Karpenter both bin-pack more effectively when the node pool isn't artificially constrained to one small instance size.

Spot and preemptible capacity for fault-tolerant workloads

Stateless, horizontally-scaled, fault-tolerant workloads — batch jobs, CI runners, async workers, most stateless HTTP services behind a load balancer — are good candidates for spot or preemptible instances, which run at a steep discount in exchange for the cloud provider being able to reclaim them with short notice. The two things that make this safe rather than a pager-duty generator are a PodDisruptionBudget that caps how many replicas can be down at once, and an autoscaler (Karpenter or Cluster Autoscaler) that reacts fast enough to replace reclaimed capacity before the PDB gets violated. Anything stateful, anything with a single replica, or anything where a 30-second interruption notice isn't enough runway does not belong on spot.

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: worker-pdb
spec:
  minAvailable: 60%
  selector:
    matchLabels:
      app: async-worker
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: async-worker
spec:
  template:
    spec:
      tolerations:
        - key: "karpenter.sh/capacity-type"
          operator: "Equal"
          value: "spot"
          effect: "NoSchedule"
      nodeSelector:
        karpenter.sh/capacity-type: spot

Cleaning up what nobody is watching

Past the scheduling math, the other reliable source of waste is things that were spun up for a project that ended and never got torn down: idle namespaces from a proof-of-concept six months ago, orphaned PersistentVolumeClaims left behind after a StatefulSet was deleted but its PVCs weren't, and LoadBalancer-type Services that provisioned a cloud load balancer nobody's pointed DNS at anymore. None of these show up in CPU/memory dashboards because they're not compute-bound costs — they're forgotten storage and forgotten cloud LB line items that accumulate quietly. A periodic `kubectl get pvc --all-namespaces` cross-referenced against active StatefulSets, and a look at your cloud provider's load balancer list against your cluster's actual Service objects, catches most of it.

Wrapping up

Cost optimization in Kubernetes isn't mainly about picking cheaper instance types — it's about closing the gap between what you've asked the scheduler to reserve and what your workloads actually use. Requests drive that reservation and the bill; limits only govern what happens once a pod is already running. Right-size requests from real usage data first, bin-pack onto fewer larger nodes second, move genuinely fault-tolerant workloads to spot capacity third, and sweep for orphaned PVCs and unused load balancers on a schedule. In that order, because the first one is usually where most of the money actually is.