system-design

Part 10 — Performance & Scaling Playbooks

The concrete techniques for finding and fixing bottlenecks, and the through-story that ties every tool in this repo into one ordered journey from one user to a billion.

Time for this part: Weeks 3-4 on the Standard track (the capstone chapter is Week 3; the bottleneck and scaling chapters reinforce Parts 1-2). This part is where the individual components become a strategy.

The unifying idea: measure before you optimize, do less work before doing more work faster, and apply each scaling technique only when the numbers demand it.

---

Chapters

Diagnosis

#	Chapter	Time	The problem it solves
1	How to Find a Bottleneck	18 min	“It’s slow” is not a diagnosis — measure, don’t guess

The two directions

#	Chapter	Time	The problem it solves
2	Scaling Reads	16 min	The easy direction — you can copy data freely
3	Scaling Writes	16 min	The hard direction — one source of truth

The hard cases

#	Chapter	Time	The problem it solves
4	Hot Keys and Celebrity Problems	16 min	Even sharding distributes keys, not load
5	Thundering Herd & Cache Stampedes	14 min	Synchronized demand hitting a resource at once
6	Connection Pooling & Resource Limits	14 min	Bounding every resource

The whole story

#	Chapter	Time	The problem it solves
7	1 User → 1 Billion Users ⭐	25 min	Every technique, in the order you’d apply them

---

The things to remember

1. Measure before optimizing — intuition about bottlenecks is usually wrong, and Amdahl’s Law means optimizing anything but the dominant cost is wasted. Identify the bound (CPU/IO/memory/network).
2. Reads scale by copying (cache, replicas, CDN, precompute); writes don’t — eventually you batch, async, use LSM, or shard. Compute the read/write ratio first.
3. Consistent hashing distributes keys, not load. Hot keys need replication, in-process caching, CDN, or hybrid fan-out — not better hashing.
4. Thundering herds are synchronized demand — de-synchronize (jitter, backoff) or collapse the duplicate work (coalescing, stale-while-revalidate).
5. Bound every resource. The 50-servers × 100-connections outage is real; PgBouncer multiplexes thousands of client connections onto a few hundred real ones.
6. Scaling is an ordered journey. Each stage solves the previous stage’s bottleneck; each buys ~10× and adds complexity; almost every stage is about the database. Don’t skip ahead — premature scaling is the bigger risk than under-engineering.

---

Before moving on

You should be able to answer these without notes:

• Explain “measure before optimize” and how Amdahl’s Law reinforces it.
• Distinguish a p50-high problem from a p99-only problem, with causes.
• List the read-scaling ladder and the write-scaling ladder.
• Explain why hot keys aren’t solved by even sharding, and name three fixes.
• Explain the cache stampede fix (coalescing) and the avalanche fix (jitter).
• Explain the 50-servers × 100-connections outage and how PgBouncer fixes it.
• Narrate the 1-to-a-billion scaling journey in order, naming what breaks at each stage.

---

Next: Part 11 — The Interview — how to actually run a design round: the framework, the estimation, the deep dives, and the mistakes that lose points.

This site is open source. Improve this page.