Envoy as API Gateway — A Field Guide

Envoy started as Lyft's service-mesh sidecar, but its listener/filter-chain/cluster model turned out to be just as good a fit for the edge as it is for east-west traffic — which is why it ended up as the data plane inside Istio, Gloo, Contour, and a fair number of hand-rolled gateways. Running it directly as an API gateway means dealing with its config model on its own terms: static bootstrap config for the parts that don't change, and xDS for the parts that do.

Listeners, routes, and clusters

Envoy's config has three layers that matter for a gateway. A listener binds a port and decides what filter chain handles the connection (HTTP, TCP, TLS termination). A route configuration inside the HTTP connection manager filter matches incoming requests — by host, path, header — and decides which upstream to send them to. A cluster is that upstream: a named group of endpoints with a load-balancing policy and health-check config. The mental model is: listener accepts the connection, route picks the cluster, cluster picks the endpoint.

For a small, static gateway you can write all three by hand in one YAML file. For anything that changes often — new backend added, new route deployed — you want xDS instead of editing YAML and restarting.

static_resources:
  listeners:
  - name: ingress
    address: { socket_address: { address: 0.0.0.0, port_value: 8443 } }
    filter_chains:
    - filters:
      - name: envoy.filters.network.http_connection_manager
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          stat_prefix: ingress_http
          route_config:
            name: local_route
            virtual_hosts:
            - name: api
              domains: ["api.example.com"]
              routes:
              - match: { prefix: "/orders" }
                route: { cluster: orders_service, timeout: 2s }
  clusters:
  - name: orders_service
    type: STRICT_DNS
    lb_policy: ROUND_ROBIN
    load_assignment:
      cluster_name: orders_service
      endpoints:
      - lb_endpoints:
        - endpoint: { address: { socket_address: { address: orders.internal, port_value: 8080 } } }

xDS and dynamic configuration

xDS is the family of discovery APIs (LDS, RDS, CDS, EDS — listener, route, cluster, endpoint discovery service) that let a control plane push config changes to Envoy over gRPC without a restart. This is what separates "Envoy as a static reverse proxy" from "Envoy as a real gateway platform": a control plane like Istio's istiod, Gloo Edge, or a hand-rolled control plane speaking the xDS protocol can add a new cluster or shift traffic weights and every connected Envoy instance picks it up within seconds. Envoy's own config for this is just a bootstrap file pointing at the control plane's gRPC address — the interesting logic lives in the control plane, not in Envoy itself.

Rate limiting

Envoy supports both local rate limiting (per-Envoy-instance token bucket, no external dependency) and global rate limiting (a gRPC call out to a rate-limit service, typically Envoy's own ratelimit reference implementation backed by Redis, so limits are enforced consistently across a fleet of gateway instances). Local rate limiting is cheap and fine for coarse protection against a single noisy client hitting one instance; global rate limiting is what you need if the limit is "1000 requests/minute per API key" and you're running more than one Envoy replica, since local buckets would let each replica give the client its own separate quota.

mTLS and transport security

Terminating TLS at the listener is one job; using Envoy for mutual TLS between the gateway and backends (or between mesh sidecars) is another, and it's where Envoy's SDS (secret discovery service) earns its keep — certificates and keys are fetched from a secret provider at runtime instead of being baked into static config and rotated by redeploying. For north-south gateway traffic, terminate client TLS at the listener with a standard DownstreamTlsContext; for gateway-to-backend mTLS, an UpstreamTlsContext on the cluster with client cert validation gives you workload identity without the backend needing its own TLS termination logic.

Observability

Envoy emits detailed stats (per-cluster request counts, upstream latency histograms, circuit-breaker trip counts) to StatsD, or a Prometheus-format endpoint, and supports distributed tracing via OpenTelemetry, Zipkin, or Jaeger exporters configured on the HTTP connection manager. The access log format is fully configurable per listener, which matters at a gateway more than anywhere else in the mesh — this is the one place that sees every request, so getting the log fields right (upstream cluster, response flags, duration) up front saves a lot of grep-through-JSON later when triaging a latency spike.

Envoy's learning curve is mostly the config model, not the runtime — once listeners, routes, and clusters click, xDS is just "the same three things, pushed instead of loaded from disk." Start with static config for a single service, add a control plane and dynamic discovery only once you have enough backends that hand-editing YAML on every deploy actually hurts.