Acumatica · Gcp

GCP Cost Optimisation — A Field Guide

GCP Cost Optimisation — A Field Guide is the work that turns a deploy into a system. The deployment is one moment; the system is the next 18 months of uptime, incidents, and.

John Kihiu12 min read

Google Cloud's cost problems tend to look different from AWS's. BigQuery on-demand queries scanning far more data than intended, Compute Engine instances left running past a demo, and GKE clusters overprovisioned "to be safe" are the recurring offenders. The fixes are mostly configuration, not architecture — which is exactly why they get skipped.

BigQuery is usually the biggest line item

On-demand BigQuery pricing bills per byte scanned, not per query run, which means a query without a partition filter on a large table can cost far more than expected — and it will succeed and return correct results while doing it, so nothing alerts you. Partitioning tables by date and clustering by commonly filtered columns lets the query planner skip scanning irrelevant data. For teams running BigQuery constantly, capacity-based pricing (flat-rate slots) can be materially cheaper than on-demand once query volume crosses a threshold, but it requires actually forecasting slot usage rather than defaulting to on-demand indefinitely.

SELECT * is the single most common BigQuery cost bug

BigQuery is columnar — a query only pays for the columns it reads, not the whole row. SELECT * on a wide table scans every column whether the query needs it or not. Selecting only the needed columns is often a bigger cost win than any pricing model change.

Compute Engine rightsizing and commitments

GCP's recommender tooling (visible in the console under Recommendations) surfaces underutilized VMs based on actual CPU and memory usage history, and it's usually right — most overprovisioning comes from sizing for a peak that happens rarely or never. For steady-state workloads, committed use discounts trade a 1- or 3-year commitment for a meaningful discount over on-demand pricing; for workloads that can tolerate interruption, preemptible and Spot VMs cut compute cost sharply and are a good fit for batch jobs, CI runners, and fault-tolerant pipelines.

GKE: the cluster is rarely the problem, the pods are

A Kubernetes cluster's cost is driven by node pool sizing, and node pool sizing is driven by pod resource requests — teams that set generous CPU/memory requests "to be safe" force the cluster autoscaler to provision more nodes than the workload actually needs. GKE's cluster autoscaler and node auto-provisioning help, but only if pod requests reflect real usage; look at actual utilization versus requested resources before assuming the cluster needs to be bigger. Separate node pools for latency-sensitive versus batch workloads, with Spot VMs for the latter, is a common and effective split.

YAML · REALISTIC POD RESOURCE REQUESTS
resources:
  requests:
    cpu: "250m"
    memory: "256Mi"
  limits:
    cpu: "500m"
    memory: "512Mi"
# Set requests from observed p95 usage, not a round number picked
# at design time — the autoscaler provisions capacity based on
# requests, not actual usage.

Storage class and lifecycle policies

Cloud Storage buckets accumulate objects that are rarely or never re-read — logs, backups, old build artifacts — and staying on Standard storage for all of it is a common, invisible cost. Lifecycle rules that transition objects to Nearline, Coldline, or Archive storage after a defined age, and delete them after a retention period, usually require one bucket configuration change and no application code changes at all.

Labels are the precondition for any of this

None of the above is actionable without knowing which team or service is generating the spend. GCP resource labels, consistently applied at creation time and exported into BigQuery via billing export, are what make cost attribution possible — a query against the billing export table, grouped by label, turns "the GCP bill went up" into "the ml-training label went up 40% this month," which is the difference between reacting to a number and understanding it.

AreaHighest-leverage fix
BigQueryPartition/cluster tables, avoid SELECT *
Compute EngineRightsize via recommender, commit steady-state workloads
GKESet pod requests from observed usage, not guesses
Cloud StorageLifecycle rules to colder storage classes
AttributionConsistent labels + billing export to BigQuery

Wrapping up

GCP cost optimisation is mostly configuration discipline: partition BigQuery tables and stop scanning full columns you don't need, size Compute Engine and GKE to observed usage instead of guessed peaks, push cold storage to cheaper tiers, and label everything so the billing export can actually tell you where the money went.

John Kihiu
Acumatica ERP Developer · Laravel Engineer

Independent software engineer in Nairobi specialising in Acumatica customisations, Laravel backends, and tax fiscalisation integrations across East and Southern Africa.