system-design

Practice Problem: Design a Live-Streaming Comments System

Prompt: Design the live comments/chat that scrolls during a livestream (a YouTube Live / Twitch / TikTok Live with millions of concurrent viewers). Viewers post short messages; everyone watching sees a fast- moving stream of comments in real time. Attempt cold for 45 minutes first.

Tests: real-time fan-out to millions of viewers on one stream (an extreme hot-key/broadcast problem), and the insight that you can’t and shouldn’t deliver every comment to everyone. Combines chat and the celebrity fan-out.

Solution outline

1. Requirements

Functional: viewers post short comments on a live stream; viewers see a real-time feed of comments; handle very popular streams (millions watching one stream).
Non-functional: 🚨 massive fan-out to one channel — one stream can have millions of concurrent viewers all receiving comments. Low latency (comments feel live). Best-effort (dropping some comments on a firehose stream is acceptable — nobody reads 10,000 comments/sec anyway). High availability.
Out of scope: the video streaming itself (separate), moderation internals (mention).

2. Estimation

A viral stream: 5M concurrent viewers, posting at peak maybe 10,000 comments/sec on that one stream. Delivering 10,000 comments/sec × 5M viewers = 5×10¹⁰ messages/sec — 🚨 physically impossible and pointless. This number forces the key insight below.

3. The key insight — you can’t (and shouldn’t) deliver every comment

🚨 Nobody can read 10,000 comments/sec, so don’t deliver them all. Sample / rate-limit the outbound comment stream per viewer (show a readable subset, e.g. a few per second, prioritizing some — popular, pinned, from followed users). This turns an impossible fan-out into a bounded one. This “it’s OK to drop on the firehose” realization is what the problem tests.

4. High-level design

flowchart TB
    Viewers -->|post comment, WebSocket| Ingest[Comment Ingest]
    Ingest --> Q[[Per-stream comment stream]]
    Q --> Sampler[Sampler / rate-limiter<br/>pick a readable subset]
    Sampler --> Fanout[Fan-out layer<br/>broadcast to viewers of this stream]
    Fanout -->|WebSocket/SSE| Viewers2[Viewers of the stream]
    Q --> Store[(Recent comments buffer)]

Comments ingest over persistent connections. Per stream, a sampler selects a readable subset, and a fan-out layer broadcasts that subset to all viewers connected to that stream.

5. Deep dives

Fan-out to millions on one stream: this is the celebrity problem at the extreme. Viewers of a stream are grouped; use a hierarchical/tree fan-out — the sampled comment stream is pushed to many broadcast servers, each holding a subset of that stream’s viewer connections, so no single server broadcasts to 5M sockets. Connection servers hold viewer WebSockets (the chat connection model), grouped by stream.
Sampling strategy: rate-limit outbound to a human-readable pace; prioritize (pinned/creator/ moderator/superchat comments always shown; sample the rest). Different viewers can see different samples — consistency isn’t required.
Late joiners: on joining, load a small buffer of recent comments (not full history) so the feed isn’t empty.
Moderation: filter spam/abuse before fan-out (fast, best-effort); creators can pin/delete.
Ordering/consistency: loose — comments are best-effort, roughly time-ordered; exact global order and delivery aren’t needed (unlike 1:1 chat).

6. Trade-offs

7. What a strong answer includes

The realization that you must sample/rate-limit the firehose (the defining insight), hierarchical fan-out with viewers grouped by stream over persistent connections, best-effort loose-ordering guarantees, priority for special comments, and a recent buffer for joiners. A weak answer tries to reliably deliver every comment to every viewer in order — which doesn’t scale and isn’t even useful.