API Gateway & Service Mesh

Break a monolith into microservices and a new problem appears: every service now needs the same boring-but-critical machinery — authentication, rate limiting, TLS, routing, retries, metrics. Re-implementing that in every service is wasteful and inconsistent. Two patterns extract it into infrastructure: the API gateway handles traffic coming into the system from clients, and the service mesh handles traffic between services. Together they let your service code focus on business logic.

The Problem: Cross-Cutting Concerns

In a microservices architecture, a set of concerns applies to every service: verifying who's calling (authn/authz), limiting abusive traffic, terminating TLS, routing requests, retrying transient failures, and emitting metrics/traces. If each team bakes these into their service, you get duplicated effort and — worse — inconsistency: ten slightly different rate limiters, five auth implementations, uneven observability. The fix is to lift these concerns into a shared layer. Where that layer sits depends on the direction of the traffic.

North-South vs East-West Traffic

A useful mental model from networking:

• North-south traffic crosses the boundary of your system — external clients calling in (and responses going out). The API gateway sits here, at the edge.
• East-west traffic flows between your internal services. The service mesh governs this.

The gateway is the guarded front door; the mesh is the internal road network. They're complementary, not competing.

The API Gateway

An API gateway is a single entry point that sits in front of your backend services. Clients talk only to the gateway, which authenticates and routes each request to the right service and applies edge policies. Its responsibilities:

• Routing — map incoming paths/hosts to backend services.
• Authentication & authorization — validate tokens/keys once at the edge (often with OAuth/JWT) so services trust the gateway.
• Rate limiting & throttling — protect backends from abuse and overload.
• TLS termination — handle HTTPS at the edge.
• Request aggregation — combine calls to several services into one client response (reducing chatty mobile round-trips).
• Protocol translation — e.g. expose REST externally while talking gRPC internally.

clients ─▶ ┌─────────────── API GATEWAY ───────────────┐
           │ TLS term · authn · rate limit · routing   │
           └───┬──────────────┬──────────────┬─────────┘
               ▼              ▼              ▼
          order-svc      user-svc       payment-svc
   (services trust the gateway; they don't each re-auth the client)

The BFF Pattern

A web app, a mobile app, and a partner API often want different shapes of data from the same backends. The Backend for Frontend (BFF) pattern runs a separate gateway per client type, each tailoring aggregation and response shaping to that client's needs — so the mobile BFF can return a compact, battery-friendly payload while the web BFF returns a richer one, without bloating the backend services.

The Service Mesh

The gateway handles the edge, but inside a large system, services call each other constantly, and those calls need the same reliability and security features. A service mesh provides them without changing application code, using the sidecar pattern: a small proxy (e.g. Envoy) is deployed alongside every service instance, and all the service's network traffic flows through its sidecar. The sidecars (the data plane) are configured centrally by a control plane (e.g. Istio, Linkerd).

          ┌── control plane (config, policy, certs) ──┐
          │              Istio / Linkerd               │
          ▼                                             ▼
   ┌─ service A ─┐   mTLS, retries, LB, metrics   ┌─ service B ─┐
   │  app  │ proxy│◀══════════════════════════════▶│ proxy │ app │
   └───────┴──────┘   (all A↔B traffic via sidecars)└──────┴─────┘

   app code makes a plain call to "B"; the sidecar does the rest

Because every call goes through a sidecar, the mesh transparently provides: mutual TLS (encrypt + authenticate service-to-service automatically), retries, timeouts, and circuit breaking, client-side load balancing, traffic shifting (for canary deploys), and uniform observability (every hop is traced/measured — see observability). The application stays oblivious; it just calls "service B" and the sidecar handles encryption, retries, and routing.

Gateway vs Mesh

Trade-offs and When to Use Which

Both add an extra network hop and moving parts, so they're not free. An API gateway is almost always worth it once you have more than a couple of services exposed to clients — centralizing auth and rate limiting alone justifies it (just don't let it become a bloated monolith of business logic; keep it to cross-cutting concerns). A service mesh is heavier: the sidecar adds latency to every internal call and the control plane is real operational complexity. It pays off at scale — dozens-plus services where you want uniform mTLS, retries, and observability without touching every codebase — but for a handful of services it's overkill, and a few good libraries plus the gateway suffice. The honest interview answer: adopt a mesh when the cross-cutting needs across many services outweigh its operational cost, not by default.

Pitfalls

• Gateway as a monolith — stuffing business logic into the gateway recreates the monolith you split up; keep it to cross-cutting concerns.
• Gateway as a SPOF — it's on the critical path for all traffic; run it highly available and behind load balancers.
• Premature mesh — adopting a service mesh for five services adds more complexity than it removes.
• Latency blindness — extra hops (gateway + two sidecars) add up; measure the tail latency they introduce.