system-design

CDN — Content Delivery Networks

You cannot beat the speed of light, so you move the data closer instead. The single highest-impact change available for a globally-used product.

Prerequisites: Networking 101, HTTP & TLS Time to read: ~18 minutes

The problem

Your servers are in Virginia. Your user is in Karachi.

Round trip Karachi → Virginia:   ~230 ms   (physics; not negotiable)
A page needs: DNS + TCP + TLS + HTML + 40 assets

Cold load, HTTP/1.1-ish behaviour:
  DNS          ~50 ms
  TCP          230 ms
  TLS          230–460 ms
  HTML         230 ms
  Assets       several more round trips
  ────────────────────────────
  Well over 2 seconds — with a server that responds in 5 ms

99% of that time is distance. No code change touches it.

And there’s a second problem: bandwidth. Serving 2 MB images at 1 Gbps caps one server at ~62 requests/second (Latency Numbers). A viral post would saturate your entire fleet with images alone.

A CDN solves both, and it’s usually the cheapest large win in a system design.

How it works

A CDN is a global fleet of caching reverse proxies — PoPs (Points of Presence) — in hundreds of cities. Users hit the nearest one. It serves from cache, or fetches from your origin and caches for next time.

flowchart LR
    U1[User in Karachi] --> P1[PoP Karachi<br/>~10 ms]
    U2[User in Lagos] --> P2[PoP Lagos<br/>~15 ms]
    U3[User in Berlin] --> P3[PoP Frankfurt<br/>~8 ms]
    P1 -.cache miss.-> S[Shield / regional cache]
    P2 -.cache miss.-> S
    P3 -.cache miss.-> S
    S -.miss.-> O[(Origin<br/>Virginia)]

Users are routed to the nearest PoP by anycast — the same IP address announced from every location, with BGP routing each user to the closest one. Not DNS. This matters: anycast reroutes in seconds when a PoP fails, with no TTL to wait out. → DNS

📐 What it actually buys you:

Without CDN:  230 ms RTT × 4 round trips = ~900 ms + transfer
With CDN:      10 ms RTT × 4 round trips =  ~40 ms + transfer

Cache hit rate 95% → origin sees 5% of traffic
Bandwidth: the CDN serves the terabytes, you serve the misses

The TLS handshake benefit is underrated. Even for content the CDN can’t cache, terminating TLS 10 ms away instead of 230 ms away saves 2–3 round trips (~500 ms) on every new connection. The CDN then reuses a warm, long-lived connection to your origin. This is why putting dynamic content behind a CDN still helps.

What to cache — and how

Static assets: easy, do it always

Images, CSS, JS, fonts, video. Immutable and identical for everyone.

The pattern that makes this work is content hashing:

/static/app.a3f5c9e2.js       ← the hash is in the filename
Cache-Control: public, max-age=31536000, immutable    ← cache for a year

Deploy new code → new hash → new URL → the CDN fetches it fresh. You never invalidate anything, because the URL changes when the content does. This is the single most useful caching pattern in web development and it’s worth stating explicitly in an interview.

User-generated media: yes, with care

Profile photos, uploaded images, video. Cache aggressively; use versioned URLs when they change.

For private media (a photo only friends can see), use signed URLs — a time-limited, cryptographically signed link the CDN validates without calling your origin:

https://cdn.example.com/photo/abc.jpg?expires=1735689600&signature=a3f5...

The CDN checks the signature and expiry itself. Your origin authorizes once when issuing the URL, not on every fetch. This is how S3 pre-signed URLs and CloudFront signed URLs work, and it’s the standard answer to “how do you serve private media at scale?”

API responses: sometimes, and this is the interesting one

Anything identical across users and tolerant of staleness:

Cache-Control: public, max-age=60, stale-while-revalidate=300

stale-while-revalidate is excellent and underused: serve the stale copy instantly while refreshing in the background. Users never wait for a cache miss.

🚨 The classic catastrophe: caching a personalized response at a shared cache. One user’s account page gets cached and served to everyone.

Rules that prevent it:

Cache-Control: private or no-store for anything user-specific.
Vary: Authorization, Cookie if the response genuinely differs by header.
Better: put user-specific data on a separate path that’s never cached (/api/me/*), so a misconfiguration can’t leak.

Never cache

Anything with Set-Cookie for a session, anything containing PII at a shared cache, payment flows, admin interfaces.

Invalidation

The hard part, as always with caching.

Method	Speed	Notes
TTL expiry	Bounded by TTL	Simplest; just wait. Good default.
Purge by URL	Seconds–minutes	Explicit API call. Works, doesn’t scale to thousands of URLs.
Purge by tag/surrogate key	Seconds	Tag responses (`Surrogate-Key: product-42 category-shoes`), purge by tag. The best mechanism — Fastly popularized it.
Purge everything	Minutes	Nuclear. Every PoP refills from origin at once — a self-inflicted thundering herd. Avoid.
Versioned URLs	Instant	No invalidation at all. Best when you can use it.

🎙️ “I’d use content-hashed URLs for static assets so no invalidation is ever needed, and surrogate-key tagging for API responses so updating a product purges exactly the pages that reference it.”

Origin protection

A CDN doesn’t just accelerate; it shields.

Cache miss consolidation (request collapsing). 10,000 users request an uncached video segment simultaneously. Without collapsing, the PoP makes 10,000 origin requests. With it, one request goes to origin and 9,999 wait for the result. Every serious CDN does this, and it’s the reason a CDN survives a traffic spike that would kill your origin.

Tiered caching / shield PoPs. Instead of 300 PoPs each missing to origin, misses go to a small number of regional shields first. Origin sees far fewer requests, and the shield’s hit rate is much higher because it aggregates demand from many PoPs.

Origin cloaking. Lock your origin’s firewall to the CDN’s IP ranges only. Now a DDoS can’t target you directly — it has to go through infrastructure built to absorb it.

🚨 A common real-world mistake: deploying a CDN but leaving the origin publicly reachable. Attackers find the origin IP (via DNS history, certificate transparency logs, or an unproxied subdomain like mail.example.com) and bypass all your protection. If you use a CDN for DDoS protection, the origin must be unreachable except through it.

Edge compute

Modern CDNs run your code at the PoP: Cloudflare Workers, Lambda@Edge, Fastly Compute.

Genuinely good uses:

A/B test bucketing and feature flags (no origin round trip)
Auth token validation — reject bad requests 10 ms from the user instead of 230 ms away
Personalizing a cached page (fetch the shell from cache, inject the username at the edge)
Image resizing on the fly (?width=400), so you store one original
Redirects, header manipulation, bot filtering, geo-routing

Constraints: short execution limits, small memory, no persistent local state, limited language support (usually JS/WASM), and cold starts (though far smaller than Lambda’s). It’s for request shaping, not for your application.

⚖️ Edge compute genuinely improves latency, but it’s also a new place for logic to hide. Debugging “why did this behave differently for that user” gets harder when the answer is in a worker running in Lagos.

⚖️ Trade-offs

	Gain	Cost
CDN for static	Massive latency and bandwidth wins; usually cheaper than origin egress	Almost none. Do it.
CDN for API responses	Fewer origin requests, lower latency	Staleness; real risk of caching private data
Long TTLs	Higher hit rate, less origin load	Slow to update; needs an invalidation strategy
Purge-all	Simple	Thundering herd on origin; refills every PoP
Edge compute	Sub-10 ms logic near users	Execution limits; logic scattered across the edge
Multi-CDN	Redundancy; no single vendor outage	Complexity, cost, inconsistent behaviour between vendors

In the real world

Netflix Open Connect goes beyond a normal CDN: they place their own appliances, full of SSDs, inside ISPs’ networks. Popular content is pushed to them overnight during off-peak hours. At their scale (a meaningful percentage of global internet traffic), even a normal CDN isn’t close enough — the physics forces content into the ISP itself.
Cloudflare’s anycast network announces the same IPs from hundreds of cities. It’s the standard example of anycast-based routing and DDoS absorption: attack traffic is spread across the whole network rather than concentrated on one target.
Fastly’s instant purge (surrogate keys, sub-second global purge) is what makes CDN caching viable for news and dynamic sites, where content changes constantly and stale content is a real problem. Before tag-based purging, dynamic sites mostly couldn’t use CDN caching properly.

🚨 Interview traps

Forgetting the CDN entirely in any design with images, video, or global users. It’s usually the first thing to mention.
Only mentioning static assets. Dynamic content benefits from edge TLS termination and warm origin connections even at a 0% hit rate.
Not having an invalidation story. “We’ll cache for a day” invites “and how do you update it?”
Caching personalized responses at a shared cache. A security bug, not just a correctness one.
Ignoring the origin shield / request collapsing. It’s how a CDN survives a spike, and it’s a nice depth point.
Leaving the origin publicly reachable while claiming CDN-based DDoS protection.

🎙️ Soundbites

“Images and video go to object storage behind a CDN, never through the app servers — at 2 MB per image, one server’s NIC caps out around 60 requests per second.”
“Static assets get content-hashed filenames and a one-year immutable TTL, so we never invalidate anything: a new deploy is a new URL.”
“Even for uncached API responses, the CDN helps — TLS terminates 10 ms from the user instead of 230, and the edge keeps a warm connection to origin.”
“For private media I’d use signed URLs with a short expiry, so the CDN validates access without calling our origin on every fetch.”
“I’d lock the origin firewall to the CDN’s IP ranges — otherwise an attacker who finds the origin IP bypasses all of it.”

🛠️ Try it

1. Measure the difference. Put a static file behind a free Cloudflare account. Then:

# Uncached origin
curl -w 'total: %{time_total}s\n' -o /dev/null -s https://origin.example.com/big.jpg

# Via CDN — run twice, first is a MISS, second a HIT
curl -sI https://cdn.example.com/big.jpg | grep -i 'cf-cache-status\|age'
curl -w 'total: %{time_total}s\n' -o /dev/null -s https://cdn.example.com/big.jpg

2. See the headers that control everything.

curl -sI https://cdn.jsdelivr.net/npm/react@18/package.json | grep -i 'cache-control\|age\|x-cache'

Find a site using stale-while-revalidate and one using immutable. Read the values and work out what update strategy each implies.

3. Break it deliberately. Serve a personalized response (include a username) with Cache-Control: public, max-age=60 and no Vary. Hit it as two different users through a caching proxy. Watch one user see the other’s name. This is a five-minute exercise that will make you paranoid about Vary forever, which is the correct state to be in.

Check yourself

1. How does a CDN help a page whose content is completely dynamic and uncacheable?

Two ways. First, TLS and TCP terminate at a PoP ~10 ms from the user instead of ~230 ms away, saving 2–3 round trips (~500 ms) on connection setup. Second, the PoP maintains a warm, already-established connection to your origin, so the request travels over an existing connection with a fully-ramped congestion window rather than paying handshake and slow-start costs. You also get anycast routing over an optimized backbone, which is often faster than public internet routing. Hit rate can be zero and it still helps substantially.

2. Why is content hashing in filenames better than cache invalidation?

Because it eliminates the invalidation problem entirely. `app.a3f5c9e2.js` and `app.b7d2f1a8.js` are different URLs, so the CDN never needs to be told anything — a deploy just references the new URL. You can set a one-year `immutable` TTL with no risk of serving stale code, you get instant global consistency (no purge propagation delay), and rollbacks work because the old file still exists. Invalidation is the hard problem; versioned URLs sidestep it.

3. 10,000 users request the same uncached object at the same instant. What should the CDN do?

Request collapsing (cache miss consolidation): send exactly *one* request to origin and have the other 9,999 wait for that response, then serve them all from the now-populated cache. Without it, origin receives 10,000 simultaneous requests for the same object — a thundering herd that can take it down precisely when traffic is highest. Combined with tiered caching (PoPs miss to a regional shield rather than directly to origin), this is what lets a CDN absorb a viral spike.

4. What's the risk of Cache-Control: public on an authenticated endpoint?

The shared cache stores one user's response and serves it to every subsequent requester of that URL — a direct data breach. `public` explicitly tells shared caches they may store and reuse the response. Authenticated responses need `private` (browser-only) or `no-store`, and if the response legitimately varies by header you must declare `Vary: Authorization, Cookie` so the cache keys on it. Safest structural fix: keep personalized data on paths that are never cacheable, so a config mistake can't leak.

5. You put your site behind a CDN for DDoS protection, but you still get attacked. What did you miss?

The origin is still directly reachable. Attackers find origin IPs through DNS history, certificate transparency logs, unproxied subdomains (`mail.`, `ftp.`, `staging.`), email headers, or old records. Once they have it, they bypass the CDN entirely. Fixes: firewall the origin to accept traffic only from the CDN's published IP ranges (or use an authenticated origin pull / private link), make sure *every* DNS record is proxied, rotate the origin IP after enabling protection, and check certificate transparency logs for subdomains that leak it.