L4 vs L7 Load Balancing

By Michael Samuel Naeem · April 18, 2025 · 5 min read

Your API runs on four backend instances behind a load balancer. Users report intermittent 502 errors. One instance handles 80% of traffic while the others idle. WebSocket connections drop when you deploy because the load balancer routes the upgrade request to a different server than the one holding the TCP session.

The problem is usually not "load balancing is broken" — it is that you picked the wrong layer or the wrong algorithm for your traffic pattern. L4 and L7 load balancers make fundamentally different routing decisions, and the algorithm you choose determines whether traffic distributes evenly under real workloads.

OSI layers: where routing happens

Layer 4 (Transport): routes based on IP address and port. The load balancer sees TCP/UDP packets but does not inspect HTTP content.

Layer 7 (Application): routes based on HTTP properties — URL path, headers, cookies, query parameters, method.

Client request: GET /api/v2/users HTTP/1.1
                Host: api.example.com

L4 sees:  TCP to 10.0.1.5:443          → forward to backend:8080
L7 sees:  GET /api/v2/users             → route to users-service-v2 pool
          Host: api.example.com          → match virtual host
          Authorization: Bearer ...      → optional auth routing

L4 load balancing

L4 balancers operate on packets. They do not terminate HTTP — they forward TCP connections to backend servers.

How it works:

Client → L4 LB (VIP:443) → Backend A (10.0.1.10:8080)
                         → Backend B (10.0.1.11:8080)
                         → Backend C (10.0.1.12:8080)

Modes:

• Direct routing (DR): LB modifies the MAC address, backend responds directly to client. Fastest, but backends must accept traffic for the VIP.
• NAT mode: LB rewrites source/destination IP. Simpler setup, LB becomes a bottleneck.
• Tunnel mode: LB encapsulates packets. Used in cross-subnet deployments.

Pros:

• Extremely high throughput (millions of connections per second)
• Low latency (no HTTP parsing)
• Protocol-agnostic (TCP, UDP, MQTT, gRPC over TCP)

Cons:

• No path-based routing
• No TLS termination (unless using TLS passthrough with SNI)
• No HTTP health checks (TCP connect checks only)
• Sticky sessions require connection-level affinity

Examples: AWS Network Load Balancer (NLB), HAProxy in TCP mode, Linux IPVS, Cloudflare Spectrum.

L7 load balancing

L7 balancers terminate the client connection, parse the HTTP request, and open a new connection to the selected backend.

upstream api_v2 {
    least_conn;
    server 10.0.1.10:8080 weight=3;
    server 10.0.1.11:8080 weight=2;
    server 10.0.1.12:8080 weight=1;
}

server {
    listen 443 ssl;
    server_name api.example.com;

    location /api/v2/ {
        proxy_pass http://api_v2;
        proxy_set_header X-Real-IP $remote_addr;
    }

    location /api/v1/ {
        proxy_pass http://api_v1_legacy;
    }
}

Pros:

• Path, header, and cookie-based routing
• TLS termination and certificate management
• HTTP health checks (GET /health → expect 200)
• Request/response modification (headers, redirects)
• Rate limiting and WAF integration

Cons:

• Higher latency (1–5ms per request)
• Lower max throughput than L4
• HTTP-specific (gRPC works, raw TCP does not)

Examples: AWS Application Load Balancer (ALB), nginx, HAProxy in HTTP mode, Envoy, Traefik.

Load balancing algorithms

For LLM inference endpoints where requests take 2–30 seconds, least connections prevents the pile-up that round robin causes — a single slow request blocks a round-robin slot, but least connections routes around it.

Health checks: the piece everyone gets wrong

A backend that fails silently — returns 200 on /health but 500 on actual requests — passes TCP health checks but fails users.

L4 health check: TCP connect to port. Fast, but only detects crashed processes.

L7 health check: HTTP GET to a health endpoint with expected status code and optional body validation.

# HAProxy backend health check
backend api_servers
    option httpchk GET /health
    http-check expect status 200
    server api1 10.0.1.10:8080 check inter 5s fall 3 rise 2
    server api2 10.0.1.11:8080 check inter 5s fall 3 rise 2

Configure fall (consecutive failures before marking down) and rise (consecutive successes before marking up) to avoid flapping. A backend that fails 3 checks over 15 seconds goes out of rotation; it needs 2 successes to return.

Combining L4 and L7

Production setups often stack both:

Internet → L4 LB (NLB) → L7 LB (nginx/ALB) → Backend pods
           distributes    routes by path
           across AZs     terminates TLS

• L4 handles cross-AZ distribution and DDoS absorption.
• L7 handles routing, TLS, and HTTP-specific logic.
• Backend pods see clean HTTP from the L7 layer.

For Kubernetes, the Ingress controller (L7) sits behind a cloud LB (L4):

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: api-ingress
spec:
  rules:
    - host: api.example.com
      http:
        paths:
          - path: /api/v2
            pathType: Prefix
            backend:
              service:
                name: api-v2
                port:
                  number: 8080

Resources

• HAProxy configuration manual
• nginx load balancing documentation
• AWS ELB comparison (NLB vs ALB vs CLB)
• Envoy proxy load balancing policies
• Google Cloud load balancing overview