system-design

IP, TCP, and UDP

Two ways to send data across a network: one that guarantees delivery and one that doesn’t. Nearly every real-time system design hinges on knowing when to give up the guarantee.

Prerequisites: Networking 101 Time to read: ~18 minutes

The problem

The network is allowed to do horrible things to your data:

Drop it. A router’s buffer is full; your packet is discarded. No apology, no notification.
Duplicate it. A retransmission arrives after the original finally shows up. Now you have two.
Reorder it. Packet 3 takes a faster route than packet 2 and arrives first.
Delay it. A packet sits in a queue for 4 seconds and arrives after you’ve given up.
Corrupt it. A bit flips. (Checksums catch most, not all.)

You need some protocol on top of raw IP. The question is which guarantees you want, and what they cost.

IP: best-effort delivery

IP (Internet Protocol) does one job: get this packet to that address, probably.

An IP packet has a source IP, a destination IP, a TTL (hop counter, to kill packets that loop forever), and a payload. That’s essentially it. IP makes no promises about delivery, order, duplication, or timing.

This is a feature, not a flaw. Keeping the network layer dumb is why the internet scaled — routers just forward, and all the clever behaviour lives at the endpoints. (This is the “end-to-end principle,” and it’s why you can invent a new protocol without upgrading a single router.)

IPv4 vs IPv6: IPv4 addresses are 32 bits (~4.3 billion, exhausted years ago, hence NAT). IPv6 is 128 bits (effectively unlimited). For system design purposes: know they exist, know dual-stack is normal, and know that IPv6 removes the need for NAT.

Private ranges you should recognize instantly: 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16. Anything in a VPC lives here.

TCP: the reliable one

TCP turns IP’s chaos into a clean, ordered byte stream. It gives you five guarantees:

Delivery — lost segments are retransmitted until acknowledged.
Ordering — bytes arrive in the order you sent them.
Deduplication — duplicates are discarded by sequence number.
Integrity — checksums catch corruption.
Flow & congestion control — it slows down when the receiver or the network can’t keep up.

The handshake

Client                              Server
  │ ──── SYN, seq=x ─────────────────►│
  │ ◄─── SYN-ACK, seq=y, ack=x+1 ─────│
  │ ──── ACK, ack=y+1 ───────────────►│
  │        connection established      │

One full round trip before any data. → this is the cost you pay per connection, and why you pool them.

Teardown and TIME_WAIT

Closing takes a four-way exchange (FIN/ACK each direction), and the closing side then holds the connection in TIME_WAIT for ~60 seconds (2× the max segment lifetime), to make sure stray packets from the old connection don’t get confused with a new one on the same port pair.

🚨 This causes a real production problem. A busy server opening many short-lived outbound connections accumulates tens of thousands of TIME_WAIT sockets and eventually runs out of ephemeral ports (~28,000 by default). Symptom: “cannot assign requested address” under load. The fix is connection reuse (keep-alive/pooling), not tuning. Check with ss -s.

Flow control vs congestion control

Two different things people conflate:

Flow control protects the receiver. Every ACK advertises a window: “I have room for 64 KB more.” Sender respects it.
Congestion control protects the network. TCP starts slow, ramps up, and backs off hard when it detects loss (slow start, congestion avoidance, and algorithms like CUBIC or BBR).

📐 Slow start matters for latency. A new connection doesn’t start at full speed — it begins with ~10 packets (~14 KB) in flight and doubles each round trip. Transferring 1 MB over a fresh connection to a distant server takes several RTTs just to ramp up. Reusing a warm connection skips all of it. Another point for connection pooling.

Head-of-line blocking

TCP delivers bytes in order. If segment #2 is lost, segments #3–#100 sit in the receiver’s buffer, complete and useless, until #2 is retransmitted and arrives.

For a single file download, fine. For HTTP/2 — which multiplexes many independent requests over one TCP connection — one lost packet stalls every concurrent request. This is the specific flaw that motivated HTTP/3 moving to UDP. → HTTP Versions

UDP: the fast one

UDP adds almost nothing to IP: source port, destination port, length, checksum. Send a datagram, hope it arrives.

No handshake (send immediately — zero setup latency), no retransmission, no ordering, no congestion control, no connection state (so one server can serve millions of clients without per-connection memory).

When “unreliable” is the right choice

The insight beginners miss: for real-time data, a late packet is worse than a lost one.

In a voice call, if 20 ms of audio is lost, you want the next 20 ms immediately. TCP would stall the stream to retransmit audio that is now historical. UDP just moves on, and the codec conceals the gap. Nobody notices a 20 ms dropout; everybody notices a 300 ms stutter.

Same for live video, multiplayer game state (position updates supersede each other — the newest is all that matters), and metrics (losing one datapoint out of a million is noise).

Use UDP for	Why
Voice/video calls (WebRTC)	Late data is useless; jitter is worse than loss
Live game state	Newer updates supersede older ones
DNS queries	Single small request/response; retry at the app layer is cheaper than a handshake
Metrics (StatsD)	Volume is huge, individual loss is irrelevant, and it must never block the app
QUIC / HTTP/3	Reliability reimplemented in userspace, without TCP’s constraints
Service discovery, broadcast/multicast	TCP can’t multicast at all

TCP vs UDP: the comparison table

	TCP	UDP
Connection	Handshake required (1 RTT)	None — send immediately
Reliability	Guaranteed, retransmits	None
Ordering	Guaranteed	None
Duplicates	Removed	Possible
Congestion control	Yes	No (you must implement it or be a bad citizen)
Header size	20+ bytes	8 bytes
Head-of-line blocking	Yes	No
Server state per client	Yes (memory, FDs)	None
Multicast/broadcast	No	Yes
Typical uses	HTTP, databases, SSH, email	Video/voice, gaming, DNS, QUIC, metrics

⚖️ The trade-off in one line: TCP gives you correctness and costs you latency and tail behaviour. UDP gives you control and makes correctness your problem.

🚨 Note the last row of that table carefully: “UDP is faster” is a lazy statement. UDP has lower setup latency and no head-of-line blocking. Raw throughput on a healthy network is comparable — and TCP is often better, because its congestion control avoids the collapse that naive UDP causes.

QUIC: the third option

QUIC runs a reliable, ordered, encrypted protocol on top of UDP, in userspace. It’s what HTTP/3 uses. It exists because TCP couldn’t be fixed — it’s implemented in operating system kernels and in middleboxes worldwide, so changes take a decade.

What QUIC gets you:

1-RTT connection setup (0-RTT for resumed connections) — transport and TLS handshake combined.
No head-of-line blocking across streams — independent streams are genuinely independent, so a lost packet only stalls the stream it belonged to.
Connection migration — the connection is identified by a connection ID, not the IP/port tuple. Switch from Wi-Fi to cellular and your download keeps going. TCP would have to start over.
Faster protocol evolution — it ships in the application, so it updates when your browser does.

Cost: more CPU (userspace, less kernel/NIC offload), and some corporate networks block or throttle UDP.

In the real world

WebRTC (video calls in the browser) uses UDP for media and layers its own retransmission and congestion control on top, tuned specifically for latency over completeness.
Google’s search and YouTube run on QUIC/HTTP/3 for a large share of traffic; published results showed meaningful reductions in video rebuffering and search latency, especially on lossy mobile networks.
DNS uses UDP for normal queries (one packet each way — a handshake would double the cost) and falls back to TCP for large responses like DNSSEC or zone transfers.

🚨 Interview traps

“UDP is faster than TCP.” Too vague. Say why: no handshake, no retransmission stalls, no head-of-line blocking. Then say what you give up.
Choosing TCP for a live video stream and not noticing that retransmitting stale frames makes the experience worse.
Not knowing why HTTP/3 uses UDP. The answer is head-of-line blocking, and it’s a great chance to show depth.
Forgetting that TCP connections cost server memory. “We’ll hold 10 million WebSocket connections on one server” — at ~10 KB kernel state each, that’s 100 GB before your app allocates anything.
Ignoring TIME_WAIT / ephemeral port exhaustion when designing something that makes many short-lived outbound connections.

🎙️ Soundbites

“For the media path I’d use UDP — in a live call, a retransmitted packet arrives too late to be useful, so TCP’s reliability actively hurts us. Signalling can stay on TCP.”
“We’re multiplexing many requests over one connection, so a single packet loss would stall all of them under TCP. That’s the argument for HTTP/3 here.”
“Each of those persistent connections costs memory and a file descriptor on the server, so 1 million concurrent users means roughly N connection-holding servers — let me size that.”

🛠️ Try it

1. Watch a handshake.

sudo tcpdump -i any -n 'tcp port 443 and host example.com' &
curl -s https://example.com > /dev/null

Identify the SYN, SYN-ACK, ACK, and then the encrypted data.

2. See TIME_WAIT accumulate.

ss -s                                   # summary of socket states
for i in $(seq 1 200); do curl -s -o /dev/null http://example.com; done
ss -s                                   # watch timewait climb

3. Feel head-of-line blocking. On Linux, add artificial loss and compare a multi-request HTTP/2 page load with and without it:

sudo tc qdisc add dev eth0 root netem loss 2%
# load a page, observe
sudo tc qdisc del dev eth0 root

Check yourself

1. Why does a video call use UDP when losing data sounds obviously bad?

Because in real-time media, data has an expiry. Audio from 300 ms ago is useless even if it arrives perfectly — the conversation has moved on. TCP would stall the entire stream waiting for that stale packet, converting a tiny, imperceptible dropout into a noticeable freeze. UDP lets the codec skip it and stay current. Loss is concealed; latency is not.

2. What is head-of-line blocking and which protocols suffer from it?

One lost/delayed item at the front of an ordered queue blocks everything behind it, even though those items are ready. TCP has it at the transport layer (in-order byte delivery), so HTTP/2 — which multiplexes streams over one TCP connection — inherits it across all streams. HTTP/1.1 has a version of it at the application layer (one request at a time per connection). HTTP/3 over QUIC avoids the cross-stream case by keeping streams independent.

3. Your service opens a new HTTP connection to a downstream API for every request, at 2,000 RPS. What breaks?

Ephemeral port exhaustion via TIME_WAIT. At 2,000 connections/second held ~60 s in TIME_WAIT, you need ~120,000 ports and have ~28,000. You'll see "cannot assign requested address" errors. You're also paying a full handshake (plus TLS) per request, wasting a round trip and CPU. Fix: an HTTP client with connection pooling / keep-alive.

4. Why did QUIC get built on UDP rather than as a new transport protocol?

Because deploying a genuinely new IP-level protocol is impossible in practice — routers, firewalls, NATs, and middleboxes everywhere drop anything that isn't TCP or UDP, and kernels take a decade to update. Building on UDP means QUIC ships in userspace, inside the application, and can evolve at application speed while still traversing the existing internet.

5. When would you deliberately choose TCP even for a latency-sensitive system?

When correctness beats latency and you don't want to reimplement reliability: database connections, financial transactions, file transfer, and any request/response API. Also when you'd otherwise have to build retransmission, ordering, and congestion control yourself — that's a large amount of subtle work, and getting congestion control wrong makes you a bad network citizen who degrades everyone else's traffic too.