system-design

Design Twitter / X

Difficulty: Tier 2 Asked at: Meta, Twitter/X, Amazon, Careem, Noon, almost every mid+ loop Time budget: 45–60 min

The flagship social-media design question. Its whole soul is one decision — fan-out on write vs fan-out on read — and the “celebrity problem” that forces a hybrid. If you understand how a tweet gets from one person’s compose box into millions of timelines, you understand feed systems, and this pattern reappears in Instagram, news feed, and notifications.

Prerequisites: Caching, Message Queues, Hot Keys, Sharding

1. Requirements

Functional:

Post a tweet (text, ~280 chars, optional media).
Follow/unfollow users.
Home timeline — a feed of recent tweets from people you follow, reverse-chronological (or ranked).
View a user’s profile timeline (their own tweets).
Like, retweet, reply (mention briefly; the feed is the core).

Non-functional:

Massive read scale — reading timelines vastly outnumbers posting (read:write ~100:1 or more).
Low latency — the home timeline must load fast (< 200 ms).
High availability — eventual consistency is fine (a tweet appearing a second late is OK).
Scale — hundreds of millions of users, celebrities with 100M+ followers.

Out of scope: DMs (that’s chat), search (a separate system), trends, ads.

2. Estimation

300M daily active users, each posting ~2 tweets/day → 600M tweets/day ≈ ~7,000 writes/sec, peak ~5× → ~35K/sec.
Timeline reads: each user refreshes many times/day → ~100K–1M timeline reads/sec. 🚨 Reads dominate massively → the design optimizes reads.
Storage: 600M tweets/day × 300 bytes × 5 years ≈ ~300 TB of tweets (media separate in object storage).
The fan-out math: average user has ~200 followers; a celebrity has 100M. One celebrity tweet → 100M timeline insertions. This asymmetry is the problem.

3. API

POST /tweets            { text, mediaIds? }         → 201 { tweetId }
GET  /timeline/home?cursor=...                      → [tweets]   (the hard one)
GET  /timeline/user/{userId}?cursor=...             → [tweets]
POST /follow/{userId}   /  DELETE /follow/{userId}
POST /tweets/{id}/like  /  /retweet

Timelines are cursor-paginated (not offset) — feeds are huge and change constantly.

4. Data model

Tweets (wide-column, e.g. Cassandra): tweet_id (PK, Snowflake — time-sortable), user_id, text, media_refs, created_at. Sharded by tweet_id.
Follows / social graph: follower_id → followee_id (and the reverse). A graph DB or a sharded relational/KV store. Needs fast “who do I follow” and “who follows me.”
Home timeline cache (Redis): per-user list of recent tweet IDs — the precomputed feed.
Media in object storage + CDN.

5. The core decision: fan-out on write vs read

🚨 This is the entire question. How does a tweet reach followers’ home timelines?

Fan-out on write (push)

When you tweet, push the tweet ID into every follower’s precomputed timeline (in Redis) immediately.

✅ Reads are trivially fast — the timeline is already built; just read a list. Great for the read-heavy workload.
❌ Writes are expensive for popular users — a celebrity with 100M followers = 100M writes per tweet. The “celebrity problem.” Also wastes work for inactive followers.

Fan-out on read (pull)

When you open your timeline, pull recent tweets from everyone you follow and merge them on the fly.

✅ Writes are cheap — just store the tweet once.
❌ Reads are expensive — merging tweets from hundreds of followees on every timeline load, at 1M reads/sec, is brutal. Bad for a read-heavy system.

The hybrid (what real Twitter does) ⭐

Fan-out on write for normal users (the vast majority) — precompute their followers’ timelines. Fast reads.
Fan-out on read for celebrities (users above a follower threshold) — don’t push their tweets to 100M timelines. Instead, when a user loads their timeline, merge the precomputed part with a live pull of the few celebrities they follow.
🚨 This resolves the celebrity problem: you get fast reads for everyone without the 100M-write explosion. State the threshold as a tuning knob.

6. High-level design

flowchart TB
    Client -->|POST tweet| API[Tweet Service]
    API --> TweetDB[(Tweet store<br/>sharded by tweet_id)]
    API --> FanoutQ[[Fan-out queue]]
    FanoutQ --> FanoutW[Fan-out workers]
    FanoutW -->|normal user| Timelines[(Home timeline cache<br/>Redis, per user)]
    FanoutW -.celebrity: skip.-> Skip[No fan-out]

    Client -->|GET home timeline| TL[Timeline Service]
    TL --> Timelines
    TL -->|merge live| Celeb[Pull celebrity tweets]
    Graph[(Social graph)] --> FanoutW
    Graph --> TL

Post: store the tweet, enqueue a fan-out job. Workers look up followers and push the tweet ID into each normal follower’s Redis timeline. Celebrity tweets are not fanned out.

Read home timeline: read the precomputed Redis list, merge in a live pull of the celebrities you follow, sort by time, hydrate tweet IDs into full tweets (from cache/DB), return.

7. Deep dives

7a. Timeline storage & hydration

Store tweet IDs, not full tweets, in each timeline (compact; 800 IDs per user is tiny). On read, hydrate IDs → full tweets from a tweet cache (Redis) backed by the tweet store. This keeps timelines small and lets a tweet edit/delete reflect everywhere (single source of truth). Cap timelines (e.g. most recent 800) — nobody scrolls further; older pages fall back to a pull.

7b. The celebrity/hot-key problem

A celebrity’s tweet and profile are read by millions simultaneously — a classic hot key. Serve it from a heavily-cached/replicated path, and use the fan-out-on-read approach so you don’t do 100M writes. The hybrid split is the hot-key mitigation.

7c. Fan-out asynchronously

Fan-out happens in the background via a queue — the user’s POST returns as soon as the tweet is stored, not after 200M timeline writes. Workers process fan-out at their own pace. A slight delay before a tweet appears in all timelines is acceptable (eventual consistency). (Message Queues)

7d. Ranking (if asked)

Reverse-chronological is the simple version. A ranked feed scores candidate tweets (engagement, recency, affinity) with an ML model at read time over a candidate set — adds a ranking service between pull and render. Mention it as an extension; don’t rabbit-hole.

7e. Sharding

Shard tweets by tweet_id (Snowflake IDs are time-sortable → range or hash shard). Shard the social graph and timelines by user_id. A user’s timeline lives on one shard → single-shard read.

8. Bottlenecks & scaling further

Timeline read load → precomputed timelines in Redis (fan-out on write) + hydration cache.
Celebrity write explosion → fan-out on read for high-follower accounts (the hybrid).
Hot tweets/profiles → caching + replication of hot keys.
Fan-out worker load during a viral moment → queue buffers; autoscale workers.
Storage growth → shard tweets; media in object storage + CDN.

9. Trade-off summary

Decision	Chosen	Alternative	Why
Timeline building	Hybrid fan-out	Pure write / pure read	Fast reads without celebrity write explosion
Timeline contents	Tweet IDs + hydrate	Full tweets copied	Compact; single source of truth for edits/deletes
Consistency	Eventual	Strong	A tweet appearing a second late is fine
Fan-out timing	Async via queue	Synchronous on POST	POST stays fast; fan-out lags acceptably
Tweet store	Wide-column, sharded	Single relational DB	Write scale + horizontal growth

10. Follow-up questions

Walk through exactly what happens when a normal user tweets.

The POST hits the tweet service, which writes the tweet once to the sharded tweet store (assigning a time-sortable Snowflake ID) and immediately returns 201 to the client — the user isn't blocked. It also enqueues a fan-out job. A fan-out worker picks up the job, looks up the author's followers in the social graph, and for each *normal* follower pushes the new tweet's ID onto that follower's home-timeline list in Redis (capped at the most recent ~800). When those followers next load their timeline, the tweet ID is already there, so the read is a fast list fetch plus hydration — no per-read merging for this author. The whole fan-out is asynchronous, so a brief delay before the tweet appears everywhere is expected and fine.

Now a celebrity with 100M followers tweets. Why not do the same?

Because fanning out one tweet to 100M timelines is 100M writes, and celebrities tweet often, so pure fan-out-on-write would create a colossal, bursty write load and waste effort on inactive followers. Instead, celebrity tweets are *not* pushed to timelines at all (fan-out on read for them). When any user loads their home timeline, the system reads their precomputed timeline (containing normal followees' tweets) and separately pulls recent tweets from the handful of celebrities that user follows, then merges the two by time. Since a user follows only a few celebrities, the live pull is cheap, and we avoid the 100M-write explosion. The follower-count threshold that classifies "celebrity" is a tunable knob.

Why store tweet IDs in the timeline instead of full tweets?

Compactness and a single source of truth. A timeline of 800 tweet IDs is a few KB; 800 full tweets with text and media references is far larger and duplicated across every follower's timeline — hugely wasteful at hundreds of millions of users. Storing IDs and hydrating them from a shared tweet cache on read means each tweet exists once, so an edit or delete is reflected everywhere instantly (you don't have to rewrite it in millions of timelines), and timelines stay small and cheap to update during fan-out. The cost is an extra hydration step on read, which a tweet cache makes fast.

How does this stay consistent — what if fan-out lags?

The system is intentionally eventually consistent. When fan-out workers are backed up (a viral spike), some followers see a tweet a bit later than others — which is perfectly acceptable for a social feed; nobody requires that a tweet appear in all 200M timelines at the same instant. The tweet itself is durably stored synchronously before the POST returns, so it's never lost; only its propagation into timelines is asynchronous. This trade — strong durability of the tweet, eventual propagation to feeds — is what lets the write path stay fast and the read path stay precomputed.

How would you add a ranked (non-chronological) feed?

Insert a ranking stage between candidate generation and render. Instead of returning the merged timeline in pure reverse-chronological order, gather a candidate set (precomputed timeline + celebrity pulls + maybe some recommended tweets), then score each candidate with an ML model using features like recency, author affinity, predicted engagement, and media type, and sort by score. This runs at read time over a bounded candidate set so it stays fast, and it's an additive service — the fan-out/storage design underneath is unchanged. Keep it as a stated extension rather than the core, since chronological is the simpler, robust baseline.

11. What junior / mid / senior answers look like

Junior: stores tweets and, on timeline load, queries all followees and sorts (pure fan-out on read). Works at small scale; melts at 1M reads/sec.
Mid: recognizes read-heaviness, precomputes timelines (fan-out on write), uses a queue for async fan-out, stores tweet IDs and hydrates.
Senior: drives straight to the fan-out-write-vs-read trade-off, identifies the celebrity problem, and proposes the hybrid with a follower threshold as the resolution — then layers hot-key caching, timeline capping, sharding, and eventual-consistency reasoning, and treats ranking as a bounded extension.