API Gateway Patterns

By Michael Samuel Naeem · July 16, 2026 · 5 min read

An API gateway is the front door to your backend — and like a front door, it can either make everything smoother (centralized auth, clean routing, consistent logging) or become a bottleneck everyone queues behind. I've deployed gateways with Kong, AWS API Gateway, and custom Envoy configurations. The pattern works when it handles cross-cutting concerns that shouldn't be duplicated in every service. It fails when teams put business logic in the gateway, creating a distributed monolith that's harder to test than the services it fronts.

Core responsibilities

Client → [SSL] → [Auth] → [Rate Limit] → [Route] → [Transform] → Service
                     ↓           ↓            ↓           ↓
                   Logs     Metrics      Tracing     Cache

Routing

Path-based routing to services:

# Kong declarative config
services:
  - name: users-service
    url: http://users:8080
    routes:
      - name: users-route
        paths: ["/api/v1/users"]

  - name: orders-service
    url: http://orders:8080
    routes:
      - name: orders-route
        paths: ["/api/v1/orders"]

Clients see api.example.com/api/v1/users and api.example.com/api/v1/orders. Behind the gateway, these route to different services on different ports.

Centralized authentication

Validate tokens once at the gateway, pass user context to services:

Client: Authorization: Bearer <jwt>
Gateway: validates JWT → extracts user_id, roles
Gateway → Service: X-User-Id: 123, X-User-Roles: admin,user
Service: trusts gateway headers (internal network only)

Services don't validate tokens — they trust the gateway's headers on the internal network. This eliminates duplicate auth code across services. For JWT vs sessions trade-offs, gateways work well with short-lived JWTs.

Critical: services must reject requests without gateway-injected headers from external sources. Use mTLS between gateway and services.

Rate limiting

Per-client or per-API-key limits at the gateway:

plugins:
  - name: rate-limiting
    config:
      minute: 100
      hour: 5000
      policy: local
      fault_tolerant: true

Gateway rate limiting is coarse (per client/API key). Service-level rate limiting handles resource-specific quotas (e.g., 10 exports per day per user). Use both layers — see rate limiting algorithms.

BFF pattern

Separate gateways per client type:

Mobile App → Mobile BFF → [Users, Orders, Search]
Web App    → Web BFF    → [Users, Orders, Search, Analytics, Admin]
Partner    → Partner API → [Orders (limited)]

Mobile BFF returns compact responses:

// Mobile BFF response
{"orders": [{"id": "4521", "status": "shipped", "eta": "Jan 18"}]}

// Web BFF response (same data, more fields)
{"orders": [{"id": "4521", "status": "shipped", "eta": "Jan 18", "items": [...], "tracking_url": "...", "invoice_pdf": "..."}]}

Each BFF aggregates calls to backend services and shapes the response for its client. Mobile gets one round-trip instead of three.

When NOT to use a gateway

• Monolith: put auth/rate limiting in middleware, not a separate hop
• Internal service-to-service: use service mesh (Istio/Linkerd) for mTLS and observability
• Business logic routing: if routing depends on business rules, it belongs in a service
• Early-stage startup: a gateway adds operational complexity you don't need with one service

Gateway as bottleneck

Mitigations:

• Horizontal scaling: gateways are stateless — run multiple instances behind a load balancer
• Caching: cache public GET responses at the gateway (CDN or gateway cache)
• Async processing: gateway returns 202, client polls — for long operations
• Circuit breakers: stop routing to unhealthy services — see bulkhead pattern

Monitor gateway latency separately from service latency. If p99 gateway overhead exceeds 10ms, investigate plugin chain length.

AI/LLM gateway variant

The AI gateway pattern applies the same principles to LLM APIs: centralized auth, rate limiting, cost tracking, and model routing behind one endpoint.

Configure gateway timeouts shorter than upstream service timeouts — clients receive 504 from gateway, not hung connections waiting for origin.

Plugin chain ordering

Gateway middleware order matters:

1. Request ID / tracing
2. Authentication
3. Rate limiting
4. Request validation
5. Routing
6. Response transformation
7. Logging / metrics

Auth before rate limiting — otherwise unauthenticated traffic consumes rate limit budget. Rate limiting before routing — reject overload before backend selection.

mTLS and zero-trust

Service mesh vs API gateway division:

Use gateway for external clients, mesh for internal service-to-service — avoid duplicating auth in both layers.

Health checks and circuit breaking

# Kong upstream health check
healthchecks:
  active:
    http_path: /health
    healthy:
      interval: 5
      successes: 2
    unhealthy:
      interval: 5
      http_failures: 3

Remove unhealthy upstreams from rotation automatically — manual failover during incidents wastes minutes.

Pair with API rate limiting algorithms when implementing gateway-level throttling.

Common production mistakes

Teams get gateway patterns wrong in predictable ways:

• Skipping failure-mode rehearsal — run a game day or fault injection exercise before peak traffic, not after the first outage.
• Missing correlation context — every error path should carry request, trace, or tenant identifiers so incidents are debuggable.
• Optimizing for demo, not steady state — load tests, cache warm-up, and cold-start paths matter more than local dev latency.
• Undocumented trade-offs — if you chose speed over strict correctness (or vice versa), write that down for the next engineer.

API design for gateway patterns frustrates clients when pagination cursors expire silently, error bodies lack stable machine-readable codes, and rate limits return 429 without Retry-After headers.

Resources

• Kong Gateway documentation
• AWS API Gateway developer guide
• Envoy proxy documentation
• Microsoft Azure API Management patterns
• Rate limiting and backpressure