system-design

Load Balancers

The box that turns “a server” into “a fleet.” Also the box that’s usually the first single point of failure people forget to make redundant.

Prerequisites: Networking 101, Scalability Time to read: ~22 minutes


The problem

One server handles 5,000 QPS. You need 50,000. So you run ten servers.

Now: how does a request find one? You could give clients ten IP addresses and let them choose — but then what happens when server 4 dies? Every client needs to know. What about when you add server 11? What if server 7 is slow but not dead?

You need something in front that knows the current state of the fleet and routes accordingly. That’s a load balancer, and it does far more than “spread the traffic evenly.”


What a load balancer actually gives you

Distribution is only the first item on this list:

Function Why it matters
Distribute requests The obvious one
Health checking Automatically stops sending traffic to dead or sick instances — this is most of your MTTR improvement
Zero-downtime deploys Drain a server, update it, return it, repeat
TLS termination Handle HTTPS once, centrally, instead of per-service
A single stable address Clients see one endpoint; the fleet behind it changes freely
Retries and timeouts Retry a failed request on a different server, invisibly
Rate limiting / DDoS absorption First line of defence, before your app burns CPU
Observability One place that sees every request — traffic, latency, error rates

🎙️ If asked “why not just use DNS round-robin?”, the answer is that whole second row: DNS has no idea whether a server is alive.DNS


L4 vs L7

The most common load-balancer interview question, and it’s just “which layer does it make decisions at?”

Layer 4 (transport)

Sees IP addresses and ports. Does not read the payload. It picks a backend when the connection is established and forwards packets for the life of that connection.

Examples: AWS NLB, HAProxy in TCP mode, IPVS, Google Cloud’s TCP load balancer.

Layer 7 (application)

Terminates the connection, parses HTTP, and makes decisions per request.

Examples: AWS ALB, Nginx, HAProxy in HTTP mode, Envoy, Traefik, Cloudflare.

Choosing

Use L4 when Use L7 when
Non-HTTP protocols (databases, gRPC-over-raw-TCP, game servers) HTTP/HTTPS (i.e. most systems)
Extreme throughput, minimal latency You need path/host routing
End-to-end encryption required (passthrough) You want TLS termination, retries, or per-request balancing
Preserving the client IP matters and you can’t use proxy headers You need observability per endpoint

🎙️ “L7 for the public API, because we need path-based routing and per-request balancing across keep-alive connections. If we were fronting the database or a raw TCP protocol, L4.”

In practice you often have both: an L4 balancer at the edge for raw throughput and DDoS absorption, feeding an L7 layer that does the intelligent routing. That’s essentially what AWS NLB → ALB, or Cloudflare → your Nginx, looks like.


Algorithms

How does it choose which backend?

Algorithm How Use when
Round robin Next server in rotation Servers are identical and requests are uniform
Weighted round robin Bigger servers get more Heterogeneous fleet; also for canary rollouts
Least connections Fewest active connections wins Request durations vary a lot — usually the best default
Least response time Fewest connections + lowest latency Latency-sensitive; naturally routes around sick servers
IP hash hash(client IP) % N Crude session affinity without cookies
Consistent hash Hash onto a ring Cache locality — same key → same server. → Consistent Hashing
Random with two choices Pick 2 at random, take the less loaded Nearly as good as least-connections at a fraction of the coordination cost — used a lot at scale

🚨 Round robin’s failure mode is worth knowing. With uniform requests it’s fine. But if 1% of requests take 10 seconds and the rest take 10 ms, round robin will happily pile slow requests onto a server that’s already stuck with three of them. Least connections naturally avoids this, which is why it’s the better default for real workloads.

“Power of two random choices” deserves a mention because it’s elegant and it appears in real systems (Nginx, Envoy, and academic load-balancing literature): picking the better of two random servers gets you almost all the benefit of full least-connections tracking, without any global state. Good depth signal in an interview.


Health checks

The mechanism that turns a load balancer into a fault-tolerance device.

Passive health checks: watch real traffic. If a backend returns errors or times out, mark it down. Free, but only detects problems by failing real user requests.

Active health checks: the LB periodically probes each backend.

GET /health  every 5s
2 consecutive failures  → mark unhealthy, stop sending traffic
3 consecutive successes → mark healthy, resume

🚨 Shallow vs deep health checks — this is the interesting design decision.

# Shallow: is the process alive?
@app.get("/health")
def health():
    return {"status": "ok"}

# Deep: can it actually serve?
@app.get("/health/ready")
def ready():
    db.execute("SELECT 1")          # can we reach the database?
    cache.ping()                    # can we reach the cache?
    return {"status": "ok"}

Shallow checks miss the most common real failure: the process is running fine but its database connection pool is exhausted, so every request fails. It passes /health and keeps receiving traffic — a gray failure.

But deep checks have a serious failure mode of their own: if the shared database has a brief hiccup, every backend fails its health check simultaneously, the load balancer marks the entire fleet unhealthy, and you’ve turned a degraded database into a total outage. A momentary blip becomes a hard down.

⚖️ The resolution — and this is the answer that scores well: use both, for different purposes.

This liveness/readiness split is exactly how Kubernetes models it. → Kubernetes

Graceful shutdown is the other half. On deploy, the sequence should be: stop passing readiness → LB drains you (a few seconds) → finish in-flight requests → exit. Skipping the drain means every deploy drops requests.


Sticky sessions

Route a given user to the same backend every time, usually via a cookie the LB sets.

When you need it: WebSocket connections (they live on a specific server), in-progress multi-part uploads, and legacy apps with in-memory session state.

⚖️ What it costs you:

🎙️ “I’d avoid sticky sessions by putting session state in Redis, which keeps every instance interchangeable. The one place we can’t avoid affinity is the WebSocket layer — connections have to live somewhere — so there I’d pair it with a registry mapping user → gateway node, and route messages through pub/sub.”Scalability


Making the load balancer itself redundant

🚨 A single load balancer is a single point of failure that defeats the entire purpose. Candidates draw one box labelled “LB” in front of ten servers and don’t notice.

How it’s actually done:

Active–passive with a floating IP. Two LBs; one holds a virtual IP. If it dies, the standby claims the IP (via VRRP/keepalived). Failover in seconds. Simple; half your capacity idles.

Active–active via DNS. Multiple LB IPs in DNS; clients spread across them. Every LB works, but failover is bounded by DNS TTL — which, as covered in DNS, is slow and unreliable.

Anycast. The same IP announced from many locations; BGP routes each client to the nearest healthy one. Failover at network speed, no DNS involvement. This is how every large CDN and every serious global service does it.

Managed cloud LBs (AWS ALB/NLB, GCP LB, Azure LB) handle this for you — they’re internally redundant across availability zones. For most systems, “use the managed one” is the correct and boring answer, and saying so is not a weakness.

🎙️ “The load balancer itself needs to be redundant — I’d use a managed multi-AZ balancer, or an active-passive pair with a floating IP if we were self-hosting.”


Where load balancers appear in a design

More places than beginners draw:

flowchart TB
    U[Users] --> DNS[GeoDNS / Anycast]
    DNS --> E[Edge LB / CDN<br/>L4 + DDoS]
    E --> A[Application LB<br/>L7, path routing]
    A --> S1[Service A]
    A --> S2[Service B]
    S1 --> IL[Internal LB / service mesh]
    S2 --> IL
    IL --> S3[Service C]
    S3 --> DBP[Connection pooler<br/>PgBouncer]
    DBP --> DB[(Database)]

Client-side load balancing is worth knowing as the alternative: the client fetches the list of backends from service discovery and picks one itself. No extra network hop, no central bottleneck — but every client needs the logic, and updates propagate slowly. gRPC and service meshes do this.


⚖️ Trade-offs

Choice Gain Cost
L7 over L4 Content routing, retries, per-request balancing More CPU, higher latency, sees plaintext
Least connections over round robin Handles variable request duration Requires connection tracking
Deep health checks Detects gray failures A shared dependency blip can mark the whole fleet down
Sticky sessions In-memory state works Uneven load, state loss, disruptive deploys
Managed cloud LB Redundancy handled for you Less control, vendor lock-in, per-request cost
Client-side LB No extra hop, no central bottleneck Logic in every client; slow config propagation

In the real world


🚨 Interview traps


🎙️ Soundbites


🛠️ Try it

1. Build one. Two toy HTTP servers that print their own name, and Nginx in front:

upstream backend {
    least_conn;
    server 127.0.0.1:3001 max_fails=2 fail_timeout=10s;
    server 127.0.0.1:3002 max_fails=2 fail_timeout=10s;
}
server {
    listen 8080;
    location / { proxy_pass http://backend; }
}

Then: while true; do curl -s localhost:8080; sleep 0.2; done

Kill one backend. Watch how many requests fail before Nginx notices. Now add a proper health check and measure again. That delta is your MTTR, and seeing it as a real number changes how you think about health check intervals.

2. Break round robin. Make one endpoint sleep 3 seconds and another return instantly. Send mixed traffic through round robin, then through least_conn. Compare p99. The difference is the whole argument for least-connections.

3. Watch a bad deploy. Kill a backend with in-flight requests, without draining. Count the 502s. Then add nginx -s reload with graceful shutdown and count again.


Check yourself

1. Why is L4 load balancing problematic with HTTP/2 or keep-alive connections? L4 balances *connections*, not requests. With HTTP/2 a single long-lived connection carries thousands of multiplexed requests, so one client's entire traffic lands on one backend. Ten clients across ten servers can look perfectly balanced at the connection level while the request load is wildly skewed — and a heavy client saturates one server while others idle. L7 makes a fresh routing decision per request, which is what you actually want.
2. What's the danger of a health check that queries the database? Correlated failure. Every backend shares that database, so a brief database slowdown fails *every* health check simultaneously and the load balancer removes the entire fleet — turning partial degradation into a total outage. Mitigations: cache the dependency check result, require several consecutive failures, separate liveness (never checks dependencies) from readiness, and configure the LB to "fail open" and keep routing when all backends look unhealthy.
3. Round robin across 5 servers, but one is at 100% CPU and the others at 20%. What's happening? Round robin ignores load, so if request cost varies, distribution of *requests* doesn't mean distribution of *work*. Likely causes: that server is getting the expensive requests by chance or by affinity; it's handling a hot key or hot shard; it's on degraded hardware (a noisy neighbour); it holds long-lived connections from heavy clients; or it's stuck in GC or swapping. Switching to least-connections or least-response-time routes around it automatically — which is also a hint that your balancing algorithm should be doing the detection for you.
4. How do you deploy new code to 20 servers with zero dropped requests? Rolling deploy with graceful drain, a few instances at a time: mark the instance not-ready so the LB stops sending new requests → wait for the drain period (long enough for in-flight requests to finish, and longer than the health check interval) → stop accepting connections and finish in-flight work → deploy → wait for readiness to pass → return to the pool → move to the next batch. Add a canary stage first (one instance, watch error rates) and automated rollback on regression. → [Deployment Strategies](/system-design/09-deployment-and-infra/04-deployment-strategies.html)
5. When would you use consistent hashing in a load balancer instead of least connections? When *which* backend serves a request matters, not just how many are busy — almost always for cache locality. If each backend caches results, sending the same key to the same backend keeps hit rates high; least-connections would scatter the same key across all of them and every backend would cache everything. Same reasoning for sharded stateful services and for sticky routing that survives fleet changes. The trade-off: you give up load-awareness, so a hot key can overload one node.

Further reading