# WebSockets, SSE, and Long Polling > Getting data to a client the moment it changes. The server can't call you, so every solution is a > variation on "the client asks and waits." **Prerequisites:** [Client–Server Model](../01-foundations/07-client-server-model.md), [HTTP & TLS](../01-foundations/05-http-https-tls.md) **Time to read:** ~22 minutes --- ## The problem HTTP is request–response. The client asks; the server answers. The server has no way to start a conversation. But you constantly need server-initiated updates: a chat message arrives, a ride is matched, a price moves, a notification fires, a collaborator edits the document. And the client is unreachable — behind carrier NAT, on a phone whose IP changes, with no listening port. **The client must establish and hold the connection.** Every technique below is a different way of doing that. --- ## The four options ### 1. Short polling Ask repeatedly. ```javascript setInterval(async () => { const messages = await fetch('/api/messages?since=' + lastId).then(r => r.json()); render(messages); }, 3000); ``` ✅ Trivially simple. Works everywhere. Stateless servers — any instance serves any poll. ❌ **Latency up to the poll interval.** ❌ **Massively wasteful.** Most responses are empty. 📐 **The arithmetic that kills it:** 100,000 users polling every 3 seconds = **33,000 requests/second** — and if updates are rare, 99% return nothing. You're paying full request cost (TCP, TLS on new connections, auth, database query) to say "nothing new." **Use when:** updates are genuinely infrequent and multi-second latency is fine. Checking for app updates, refreshing a dashboard every minute. Don't dismiss it — for many features it's the right, boring answer. ### 2. Long polling Ask, and the server **holds the request open** until there's something to say (or a timeout). ```javascript async function poll() { try { const res = await fetch('/api/messages/wait?since=' + lastId); // may hang 30s if (res.ok) render(await res.json()); } catch (e) { await sleep(1000); } poll(); // immediately re-establish } ``` ✅ **Near-instant delivery** — the server responds the moment data exists. ✅ Works through every proxy and firewall; it's just HTTP. ✅ No empty responses. ❌ **One held connection per client**, so servers become connection-bound. ❌ A reconnect after every message — wasteful for high-frequency streams. ❌ 🚨 **There's a gap between responses.** A message arriving in the microsecond between the server responding and the client reconnecting can be missed — unless you use a cursor (`since=lastId`), which you must. **Use when:** you need real-time delivery but can't use WebSockets, or as a **fallback**. It's still widely deployed for exactly that reason. ### 3. Server-Sent Events (SSE) One long-lived HTTP response that the server streams into. Server → client only. ```javascript const events = new EventSource('/api/stream'); events.onmessage = (e) => render(JSON.parse(e.data)); events.addEventListener('price', (e) => updatePrice(e.data)); // Automatic reconnection is built in ``` ``` HTTP/1.1 200 OK Content-Type: text/event-stream Cache-Control: no-cache data: {"msg": "hello"}\n\n id: 42\n event: price\n data: {"symbol": "AAPL", "price": 195.3}\n\n ``` ✅ **Plain HTTP** — works with existing infrastructure, auth, and compression. ✅ **Automatic reconnection built into the browser**, including `Last-Event-ID` replay so you don't lose messages across a reconnect. This is genuinely excellent and underappreciated. ✅ Simpler than WebSockets on both ends. ❌ **One-way only.** Client→server needs a separate normal request. ❌ Text only (base64 your binary). ❌ 🚨 **Over HTTP/1.1, browsers limit ~6 connections per domain** — so 6 tabs and the seventh hangs. Over HTTP/2 this disappears (multiplexing). Worth knowing. 🚨 **SSE is under-used.** For a notification feed, a live dashboard, an activity stream, or LLM token streaming — anything one-directional — SSE is simpler than WebSockets and gives you reconnection for free. Proposing SSE where it fits, instead of reflexively reaching for WebSockets, is a good signal. ### 4. WebSockets A persistent, full-duplex connection. Starts as HTTP, then upgrades: ```http GET /chat HTTP/1.1 Upgrade: websocket Connection: Upgrade Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ== ``` ```http HTTP/1.1 101 Switching Protocols Upgrade: websocket ``` After that, it's a raw bidirectional message channel with ~2–14 bytes of framing overhead per message. ✅ **True bidirectional**, low latency, minimal per-message overhead. ✅ Binary or text. ❌ **Stateful servers** — with everything that implies (below). ❌ Not plain HTTP, so proxies and load balancers need explicit configuration. ❌ You implement reconnection, heartbeats, and message replay yourself. **Use when:** you genuinely need client→server messages at high frequency — chat, collaborative editing, multiplayer games, trading interfaces. --- ## The comparison | | Short poll | Long poll | SSE | WebSocket | | --- | --- | --- | --- | --- | | Direction | Client pulls | Client pulls | **Server → client** | **Bidirectional** | | Latency | Up to interval | Near-instant | Near-instant | Lowest | | Connections held | None | 1 per client | 1 per client | 1 per client | | Server state | Stateless ✅ | Stateful | Stateful | Stateful | | Auto-reconnect | N/A | Manual | **Built in** ✅ | Manual | | Proxy-friendly | ✅ | ✅ | ✅ | Needs config | | Overhead/message | Full HTTP | Full HTTP | ~10 bytes | ~2–14 bytes | | Complexity | ⭐ | ⭐⭐ | ⭐⭐ | ⭐⭐⭐⭐ | 🎙️ **The decision rule:** *"Do I need client→server messages at high frequency? If no, SSE. If yes, WebSockets. If the volume is low or latency tolerance is seconds, just poll."* --- ## The real cost: persistent connections make servers stateful 🚨 **This is the part that matters in a design interview**, and it's what candidates who've only read about WebSockets miss. 📐 **Capacity.** Each connection costs kernel buffers plus application state — call it ~10–50 KB. ``` 1 million concurrent connections × 30 KB = 30 GB of connection memory alone ``` Plus a file descriptor each (raise `ulimit -n`), plus per-connection heap in your runtime. Realistic per-node capacity is 50,000–500,000 connections depending on language and how much state you keep, which means **10–20 nodes for a million users** just to hold connections. **And here's the structural problem:** user A's connection lives on gateway node 3. A message for user A arrives at node 7. **How does node 7 reach user A?** ```mermaid flowchart LR A[User A] -.WebSocket.-> G3[Gateway 3] B[User B] -.WebSocket.-> G7[Gateway 7] G7 -->|1 . send to user A| PS[[Redis Pub/Sub
or Kafka]] PS -->|2 . delivered| G3 G3 -.->|3 . pushed| A R[(Connection registry
user → node)] --- G3 R --- G7 ``` **Two mechanisms are needed:** 1. **A connection registry** — `user_id → gateway_node`, in Redis with a TTL, updated on connect and disconnect. So any node can find out where a user is connected. 2. **A message bus** — the node that receives a message publishes to the channel for the target node; that node pushes over the socket. Redis pub/sub is the usual choice; Kafka if you need durability. 🚨 **Simpler variant worth mentioning:** skip the registry and have every gateway subscribe to a Redis channel per connected user (`user:42`). Publishing to `user:42` reaches whichever node has them, with no lookup. Costs more subscriptions; removes a lookup and a consistency problem. Both answers are good — knowing the trade-off is better. This routing problem is the heart of the [chat system design question](../12-case-studies/11-chat-system.md). --- ## Operational realities **Load balancers need configuration.** ```nginx location /ws { proxy_pass http://backend; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade"; proxy_read_timeout 3600s; # or the LB kills idle connections } ``` Forgetting the `Upgrade` headers or leaving a 60-second idle timeout is the single most common "WebSockets don't work in production but work locally" bug. **Deploys disconnect everyone.** Rolling a fleet of gateway nodes drops every connection on each node. At scale, all clients reconnecting simultaneously is a [thundering herd](../10-performance/05-thundering-herd.md) that can take down the new instances. **Mitigations:** drain slowly (disconnect a small percentage at a time), and **require clients to reconnect with exponential backoff and jitter**. The jitter is not optional — without it, a million clients reconnect in the same second. **Heartbeats are mandatory.** TCP connections die silently — NAT tables expire (often ~5 minutes idle), mobile networks drop, laptops sleep. Without a ping/pong every ~30 seconds, both sides hold dead connections believing they're fine. Servers leak memory; clients think they're connected and receive nothing. **Message loss across reconnects.** When a client reconnects, what did it miss? You need either: sequence numbers so the client requests everything after its last seen ID (SSE gives you `Last-Event-ID` for free), or server-side buffering of recent messages per user, or a "fetch missed messages" REST call on reconnect. **Design this explicitly** — "reconnect and hope" loses messages. **Backpressure.** A slow client (bad mobile connection) can't consume as fast as you produce. The server's send buffer grows until it OOMs. You need: bounded per-connection buffers, and a policy — drop old messages, drop the connection, or slow the producer. **Auth.** The browser `WebSocket` API can't set custom headers, so you can't send `Authorization: Bearer`. Options: a token in the query string (ends up in logs — use a short-lived one-time ticket), a cookie (works, but consider CSRF — check the `Origin` header), or authenticate as the first message after connecting. --- ## Scaling patterns **Separate the connection tier from the business logic tier.** ``` Clients → [Gateway nodes: hold connections only] → [Stateless services: business logic] ``` Gateways are dumb, memory-heavy, and scaled by connection count. Services are stateless, CPU-bound, and scaled by request rate. Deploying a business-logic change then doesn't disconnect anyone — **this alone justifies the split.** **Use sticky routing at the load balancer** so reconnects prefer the same node (warm state), but never *depend* on it. **Consider a managed service.** Pusher, Ably, AWS API Gateway WebSockets, PubNub, or Supabase Realtime handle connection management, scaling, and reconnection for you. For many products this is the right call — connection management at scale is a real engineering investment. 🎙️ *"For a million concurrent connections I'd separate the gateway tier from business logic, keep a user→node registry in Redis, and route messages through pub/sub. If this weren't core to our product, I'd seriously consider a managed realtime service instead."* --- ## ⚖️ Trade-offs | Decision | Gain | Cost | | --- | --- | --- | | Polling | Stateless, trivially simple, scales like normal HTTP | Latency; wasted requests | | Long polling | Instant delivery over plain HTTP | Held connections; reconnect per message | | SSE | Instant, simple, free reconnection + replay | One-way; HTTP/1.1 connection limit | | WebSockets | Bidirectional, lowest overhead | Stateful servers, routing layer, manual reconnect/heartbeat | | Managed service | No connection infrastructure to build | Cost, vendor dependency, less control | | Separate gateway tier | Deploys don't disconnect users | An extra tier and an internal hop | --- ## In the real world - **Slack** uses WebSockets, and has published on the reconnection storm problem — when a large number of clients reconnect at once (after a deploy or a network event), the reconnect load can exceed steady-state load by orders of magnitude. Jittered backoff is their central mitigation. - **WhatsApp famously served ~2 million connections per server** on tuned FreeBSD/Erlang. Erlang's lightweight processes ([concurrency](../01-foundations/15-concurrency-basics.md)) make per-connection overhead tiny. It's the canonical example of why runtime choice matters for connection-heavy workloads. - **LLM chat interfaces** (including this one) stream tokens over SSE, not WebSockets — the data flows one way, so SSE's simplicity and automatic reconnection win. A good current example of choosing the simpler tool. --- ## 🚨 Interview traps - **Reaching for WebSockets when SSE would do.** If it's one-directional, say so and pick SSE. - **Not sizing the connections.** A million WebSocket users is a capacity calculation, not a checkbox. - **Ignoring the routing problem.** "How does the message reach the right gateway node?" is *the* follow-up question. Answer: registry + pub/sub. - **No reconnection strategy.** Deploys and network blips disconnect everyone; without jittered backoff you get a thundering herd. - **No message-loss story across reconnects.** Sequence numbers or server-side buffering. - **Forgetting heartbeats.** Dead connections leak resources silently. - **Not mentioning stateful servers** as the fundamental cost. --- ## 🎙️ Soundbites - *"This is one-directional, so I'd use SSE rather than WebSockets — it's plain HTTP, and the browser gives us automatic reconnection with `Last-Event-ID` replay for free."* - *"A million connections at roughly 30 KB each is 30 GB of connection memory, so around 10–20 gateway nodes just to hold them."* - *"Connections make the gateway tier stateful, so I'd keep a user→node map in Redis and route messages via pub/sub to whichever node holds that user's socket."* - *"Clients reconnect with exponential backoff plus jitter — otherwise a deploy disconnects a million clients that all reconnect in the same second and take down the replacements."* - *"I'd split the connection gateway from business logic so a normal service deploy doesn't drop every user's socket."* --- ## 🛠️ Try it **1. Build all four.** The same feature — a live counter — as short polling, long polling, SSE, and WebSockets. Open 100 tabs against each and compare: requests/second in the server log, server memory, and observed latency. The numbers make the trade-offs concrete in a way no table does. **2. Find your connection ceiling.** Write a script opening WebSocket connections until something breaks. You'll hit `ulimit -n` first (raise it), then memory. Divide 1,000,000 by your result — that's how many nodes you'd need. **3. Break it through a proxy.** Run your WebSocket server behind Nginx *without* the `Upgrade` headers. Watch it fail. Add them. Then set `proxy_read_timeout 60s` and watch idle connections drop after a minute. This is the most common production WebSocket bug, and causing it once means you'll never be confused by it. **4. Simulate the reconnect storm.** Open 1,000 connections, kill the server, restart it. Watch what happens without backoff (all reconnect instantly), then with jittered exponential backoff. --- ## Check yourself
1. When would you choose SSE over WebSockets? When data flows only server→client: notification feeds, live dashboards, activity streams, progress updates, price tickers, LLM token streaming. SSE is plain HTTP (so it works through existing proxies, auth, and compression with no special configuration), it's simpler to implement on both ends, and the browser provides automatic reconnection with `Last-Event-ID` replay so you don't lose messages across a drop. WebSockets earn their extra complexity only when the client also sends frequently — chat, collaborative editing, games. The main SSE caveat is the ~6-connection-per-domain limit on HTTP/1.1, which HTTP/2 removes.
2. Your chat app has 1 million concurrent WebSocket users. How does a message reach the right one? Connections are spread across many gateway nodes, so the node receiving the message usually isn't the one holding the recipient's socket. You need two things: a **connection registry** mapping `user_id → gateway_node` (Redis with a TTL, updated on connect/disconnect), and a **message bus** — the receiving node looks up the recipient's node and publishes to a channel that node subscribes to (Redis pub/sub, or Kafka if you need durability), and that node writes to the socket. A simpler variant skips the registry: each gateway subscribes to a per-user channel for its connected users, so publishing to `user:42` reaches whichever node holds them without any lookup.
3. Why are heartbeats necessary on a persistent connection? Because TCP connections die silently. NAT tables expire after a few minutes of inactivity, mobile networks hand off or drop, laptops sleep, and intermediate proxies close idle connections — and neither endpoint is notified. Without a periodic ping/pong (typically every 20–30 seconds), the server holds connection state and memory for clients that are long gone (a slow leak that eventually OOMs), and the client believes it's connected while receiving nothing. Heartbeats detect the dead connection so the server can free resources and the client can reconnect.
4. You deploy a new version. What happens to a million WebSocket connections, and how do you manage it? Every connection on each node being replaced is dropped, and all those clients reconnect. Without care, they reconnect simultaneously — a thundering herd that can overwhelm the new instances and cause a cascading failure, plus a spike of auth and state-restoration work. Mitigations: **jittered exponential backoff** in clients (essential — never reconnect immediately); **slow drain**, disconnecting a small percentage of connections at a time rather than a whole node at once; and **separating the gateway tier from business logic** so most deploys don't touch the connection-holding tier at all. That last one is the structural fix.
5. A client reconnects after a 30-second network drop. How do you avoid losing messages? Give every message a monotonically increasing sequence number or ID per stream, have the client record the last one it processed, and on reconnect send it (`?since=`, or SSE's `Last-Event-ID` header, which the browser sends automatically). The server then replays anything after that point. This requires the server to buffer recent messages — either in memory per user with a bounded window, or by reading from a durable store like a Kafka topic or a messages table. The alternative pattern is a "sync" REST call on reconnect that fetches everything missed. What you must not do is treat the reconnect as a fresh start and hope nothing was in flight.
--- ## Further reading - [Geospatial Indexing](./21-geospatial-indexing.md) — next - [Design a Chat System](../12-case-studies/11-chat-system.md) - [Client–Server Model](../01-foundations/07-client-server-model.md) - [Thundering Herd](../10-performance/05-thundering-herd.md)