API gateways and service meshes both sit in the request path and both handle things like routing, retries, and observability — which is why teams keep asking whether they need one, the other, or both. The short answer: they solve different traffic problems, and most systems past a certain size end up running both, each scoped to the traffic it's actually good at.
North-south vs. east-west traffic
An API gateway sits at the edge, handling north-south traffic — requests coming in from clients outside your cluster: mobile apps, browsers, partner integrations. A service mesh (Istio, Linkerd, Consul Connect) sits inside the cluster, handling east-west traffic — the calls services make to each other. This is the cleanest way to draw the line: if the caller is outside your trust boundary, it's gateway territory; if both caller and callee are your own services, it's mesh territory. Routing all internal service-to-service calls back out through the edge gateway to get its features works, but it adds a network hop and makes the gateway a bottleneck for traffic volumes it wasn't sized for.
How a mesh actually works: the sidecar model
A service mesh works by injecting a sidecar proxy (Envoy, in Istio's case) next to every service instance. All traffic in and out of the service passes through its sidecar, which handles mTLS, retries, timeouts, and traffic shaping — without the service's own code knowing any of it is happening. This is the mesh's core value proposition: uniform network policy across every service, enforced by infrastructure, with zero application code changes. The cost is operational: you're now running and upgrading a proxy per instance, and debugging a request means understanding two hops (app to sidecar, sidecar to sidecar) instead of one.
Getting mutual TLS between every service pair right by hand — certificate issuance, rotation, verification — is real work most teams under-invest in. A mesh gives you it by default across the whole cluster, which is often the single strongest argument for adopting one.
Feature overlap, and where it actually ends
Both gateway and mesh do retries, timeouts, circuit breaking, and telemetry, which is where the confusion comes from. But a gateway's routing decisions are typically coarse — which service owns this path — while a mesh's routing can do fine-grained traffic splitting for canary releases, A/B tests, and per-request load balancing between service versions. A gateway generally doesn't know or care about your internal service topology; a mesh is built entirely around it.
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
name: payments
spec:
hosts: ["payments.internal"]
http:
- route:
- destination:
host: payments.internal
subset: v1
weight: 90
- destination:
host: payments.internal
subset: v2
weight: 10The operational cost a mesh adds
A mesh is not a free upgrade. Every pod gains a sidecar container, adding memory overhead and startup latency across the whole cluster. Debugging becomes harder because a failed request could be failing in your code, the sidecar, or the control plane pushing config to the sidecar. Teams below roughly a few dozen services rarely have enough east-west traffic complexity to justify this cost — a gateway plus straightforward client-side retry logic in each service covers most of what they need, and the mesh becomes worth it once you have enough services that manually keeping mTLS and retry policy consistent across all of them stops being realistic.
If the only thing you want is mTLS or uniform retries, evaluate whether a shared library or a simpler proxy pattern gets you there before taking on an entire mesh control plane and its upgrade cadence.
Running both together
In practice, larger systems run a gateway at the edge for client-facing concerns (auth, rate limiting, external API versioning) and a mesh internally for service-to-service concerns (mTLS, fine-grained routing, resilience policy). The gateway typically terminates at the mesh's ingress gateway component, which is itself mesh-managed — so the two aren't really competing, they're stacked, each doing the job it's actually built for.
| Concern | Gateway | Service mesh |
|---|---|---|
| Traffic direction | North-south (client to cluster) | East-west (service to service) |
| mTLS between services | Not its job | Core feature |
| Canary/traffic splitting | Coarse, path-based | Fine-grained, per-service |
| Operational overhead | One component to run | Sidecar per pod, control plane |
Wrapping up
Pick based on traffic direction, not feature checklists: gateway for what crosses your trust boundary, mesh for what happens between your own services once that complexity earns its operational cost. Most systems don't need a mesh on day one, and plenty never will.
Independent software engineer in Nairobi specialising in Acumatica customisations, Laravel backends, and tax fiscalisation integrations across East and Southern Africa.