system-design

Apache Kafka Deep Dive

Not a queue — a distributed, durable, replayable log. Understanding that one distinction explains everything else about it.

Prerequisites: Message Queues, Replication Time to read: ~28 minutes


The problem Kafka was built for

LinkedIn, around 2010, had a familiar mess: dozens of systems producing data (user activity, page views, metrics, database changes) and dozens consuming it (search, analytics, monitoring, recommendations, the data warehouse). Each producer–consumer pair had its own bespoke pipeline.

N producers × M consumers = N×M integrations

Every new consumer meant touching every producer. Every format change broke things nobody knew about. Adding a data warehouse meant a six-month project.

Kafka’s answer: put one durable log in the middle.

N producers → [ Kafka ] → M consumers        =  N + M integrations

Producers write once. Consumers read independently, at their own pace, from wherever they choose in the history. That’s it — and every design decision in Kafka follows from making that log fast, durable, and scalable.


The core idea: an append-only log

🧠 Mental model. A traditional queue is a to-do list — you take an item off and it’s gone. Kafka is a journal — entries are appended, never removed on read, and each reader keeps a bookmark.

Partition 0:  [0][1][2][3][4][5][6][7][8][9] ← new messages appended here
                        ↑              ↑
              analytics group      email group
              (offset 3)           (offset 8)

Consequences that make Kafka different from RabbitMQ or SQS:

🚨 If a design needs replay, or multiple independent consumers of the same stream, that’s the argument for Kafka. If it just needs work distributed to workers, SQS or RabbitMQ is simpler and you should say so.


Topics, partitions, and offsets

A topic is a named stream (user.events, orders). Each topic is split into partitions — the unit of parallelism and ordering.

Topic: user.events
├── Partition 0: [msg][msg][msg]...    → Broker 1
├── Partition 2: [msg][msg]...         → Broker 2
└── Partition 3: [msg][msg][msg]...    → Broker 3

Which partition does a message go to?

partition = hash(message_key) % num_partitions      # if a key is provided
partition = round-robin / sticky                    # if no key

🚨 The key decision: messages with the same key always land in the same partition, so they’re ordered relative to each other. Key by user_id and all events for that user are strictly ordered, while different users process in parallel.

Ordering is per-partition only. There is no global ordering across a topic, and asking for it means one partition, which means one consumer, which means no parallelism. Design around per-key ordering instead. → Message Queues

How many partitions? This is a real design question:


Consumer groups

The mechanism that gives you both queue and pub/sub semantics from one system.

flowchart LR
    T["Topic: orders<br/>P0 P1 P2 P3"]
    T --> G1
    T --> G2
    subgraph G1["Group: email-service"]
        C1[consumer 1 → P0,P1]
        C2[consumer 2 → P2,P3]
    end
    subgraph G2["Group: analytics"]
        C3[consumer 1 → P0,P1,P2,P3]
    end

🚨 A consumer group can’t have more active consumers than partitions. 4 partitions, 10 consumers → 6 sit idle. This surprises people trying to scale out, and it’s the reason partition count is a capacity decision, not a detail.

Rebalancing happens when a consumer joins, leaves, or dies: partitions are reassigned. During a rebalance, consumption pauses. Frequent rebalances (from slow processing exceeding max.poll.interval.ms) cause a nasty failure mode where the group spends its time rebalancing instead of consuming. Modern Kafka uses cooperative/incremental rebalancing to reduce the pain.


Why it’s so fast

Kafka routinely handles millions of messages per second on ordinary hardware. Four reasons, and all of them come straight from Computer Fundamentals:

1. Sequential disk I/O only. Kafka only ever appends. Sequential writes on a spinning disk reach hundreds of MB/s; random writes manage a few MB/s. By never updating in place, Kafka gets memory-like throughput from cheap disks.

2. The OS page cache does the caching. Kafka doesn’t maintain its own cache in the JVM heap. It writes to the page cache and lets the OS handle it. Consumers reading recent data — which is most consumers — are served from RAM without Kafka doing anything.

3. Zero-copy transfer. sendfile() moves data from the page cache to the network socket without passing through userspace. No copy into the JVM, no serialization, no GC pressure.

4. Batching and compression. Producers batch messages and compress the batch. Better compression ratios (similar messages compress well together) and fewer, larger network operations. The batch stays compressed on disk and is decompressed by the consumer.

🎙️ “Kafka’s throughput comes from working with the hardware rather than around it — append-only sequential writes, the OS page cache instead of an application cache, and zero-copy to the socket.” This is a great answer to “why is Kafka fast?” and it demonstrates you understand the fundamentals, not just the product.


Durability and replication

Each partition has a leader and followers on other brokers.

Partition 0:  Leader = Broker 1,  Followers = Broker 2, Broker 3

All reads and writes go to the leader; followers replicate. If the leader dies, a follower is promoted.

ISR (In-Sync Replicas) — the set of replicas caught up with the leader. A replica that falls behind is removed from the ISR and isn’t eligible for promotion.

The producer’s acks setting is the durability knob, and it’s a good interview topic:

acks Waits for Durability Speed
0 Nothing — fire and forget ❌ Messages lost freely Fastest
1 Leader only ⚠️ Lost if the leader dies before replicating Fast
all (-1) All in-sync replicas ✅ Survives broker failure Slower

🚨 acks=all alone isn’t enough. If the ISR has shrunk to just the leader, “all in-sync replicas” means one replica, and you’re back to acks=1 durability without noticing. You must also set min.insync.replicas=2 — then a write fails if fewer than 2 replicas are in sync, which is correct behaviour: refuse the write rather than silently lose durability.

📐 The standard durable configuration:

replication.factor = 3
min.insync.replicas = 2
acks = all

This survives one broker failure with no data loss, and still accepts writes with one broker down. It’s the configuration to state in an interview.

unclean.leader.election.enable=false — don’t promote an out-of-sync replica even if it means the partition is unavailable. Choosing consistency over availability (CP). The default is false for good reason; setting it true trades silent data loss for uptime.


Offsets and delivery semantics

The consumer decides when to commit its offset, and that decision determines your semantics:

# At-most-once: commit first, then process
consumer.commit()
process(msg)          # crash here → message lost

# At-least-once: process first, then commit  ← the standard choice
process(msg)
consumer.commit()     # crash here → message reprocessed (duplicate)

Kafka’s “exactly-once semantics” (EOS) uses idempotent producers (sequence numbers deduplicate retries) and transactions (atomically write to several partitions and commit offsets together).

🚨 It’s exactly-once within Kafka. A read-process-write pipeline that stays inside Kafka can be exactly-once. The moment your consumer charges a credit card or writes to Postgres, you’re outside the transaction boundary and you need idempotency again. Say this precisely — it’s a common place where candidates overclaim.


Log compaction

An alternative to time-based retention: instead of deleting old messages, keep the latest value for each key forever.

Before:  (user1, "A") (user2, "X") (user1, "B") (user3, "Z") (user1, "C")
After:   (user2, "X") (user3, "Z") (user1, "C")

Why it’s powerful: the topic becomes a durable, replayable snapshot of current state. A new service can read the compacted topic from the beginning and rebuild a complete picture without querying anyone. This is how Kafka Connect stores connector state, how Kafka itself stores consumer offsets, and how CDC topics stay bounded.

Use for: database change streams, configuration, entity state, materialized views.


Kafka’s ecosystem

KRaft — modern Kafka has removed the ZooKeeper dependency, using an internal Raft implementation for metadata. If you’re describing a current deployment, “Kafka with KRaft, no ZooKeeper” is the up-to-date answer.


When not to use Kafka

⚖️ Kafka is genuinely operationally heavy: brokers to size and monitor, partition and retention planning, consumer lag management, rebalance tuning, and a real learning curve.

Don’t use it for:

🎙️ “We need a queue, not an event log — no replay, one consumer group, modest volume. SQS gives us that with zero operational burden. I’d revisit Kafka if we needed multiple independent consumers or replay.”


⚖️ Trade-offs

  Gain Cost
Kafka over a queue Replay, multiple consumer groups, enormous throughput, long retention Real operational complexity
More partitions More parallelism More file handles/memory; slower failover; can’t reduce later
acks=all + min.insync=2 No data loss on broker failure Higher write latency; writes fail if replicas are down
Keyed messages Per-key ordering Hot keys concentrate on one partition
Log compaction A replayable snapshot of current state Loses history; only the latest value per key
Long retention Replay far back; new consumers get full history Storage cost

In the real world


🚨 Interview traps


🎙️ Soundbites


🛠️ Try it

1. Run it. docker compose with a single-broker Kafka (KRaft mode — no ZooKeeper needed anymore). Create a topic with 3 partitions:

kafka-topics --create --topic orders --partitions 3 --replication-factor 1 --bootstrap-server localhost:9092
kafka-console-producer --topic orders --property "parse.key=true" --property "key.separator=:" --bootstrap-server localhost:9092
> user1:order-A
> user2:order-B
> user1:order-C

2. See partitioning and ordering. Run two consumers in the same group, then check which partition each message landed in. Confirm that user1’s messages are always in the same partition and always in order.

3. Prove replay works. Consume everything. Then:

kafka-consumer-groups --reset-offsets --to-earliest --group my-group --topic orders --execute --bootstrap-server localhost:9092

Watch your consumer reprocess history from the start. This is the thing you cannot do with SQS, and doing it once makes the distinction permanent.

4. Find the partition limit. Create a topic with 2 partitions and start 5 consumers in one group. Watch 3 of them receive nothing. Then increase partitions and watch them wake up.


Check yourself

1. What's the fundamental difference between Kafka and a queue like SQS? Kafka is an append-only *log*: reading doesn't consume, messages persist for a retention period, consumers track their own offsets, and any consumer can rewind and reprocess history. A queue is a *to-do list*: a message is delivered, acknowledged, and deleted, with the broker tracking per-message state. This is why Kafka supports many independent consumer groups reading the same data and supports replay, while a queue supports per-message acknowledgment, selective redelivery, and message-level TTLs. Neither is better — they solve different problems.
2. Why is acks=all not sufficient for durability on its own? `acks=all` means "wait for all *in-sync* replicas." If replicas have fallen behind and been removed from the ISR, the ISR can shrink to just the leader — so "all in-sync replicas" means one, and you have `acks=1` durability while believing you have more. You must also set `min.insync.replicas=2`, which makes writes *fail* when fewer than 2 replicas are in sync. Failing the write is correct: it surfaces the degraded state instead of silently accepting data you might lose.
3. You have 4 partitions and add a 6th consumer to the group. What happens? Two consumers sit completely idle. Within a consumer group, each partition is assigned to at most one consumer, so partition count is a hard ceiling on group parallelism. Adding consumers beyond that does nothing except trigger a rebalance. To scale further you must increase partitions — but note that adding partitions changes `hash(key) % N`, so existing keys may move to different partitions, breaking ordering guarantees for messages already in flight. This is why partition count deserves thought upfront.
4. Explain why Kafka is fast in terms of hardware behaviour. Four things, all working *with* the hardware: (1) **Sequential-only I/O** — Kafka appends and never updates in place, so disk writes are sequential (hundreds of MB/s) rather than random (a few MB/s). (2) **OS page cache** — Kafka doesn't maintain a JVM-heap cache; it writes to the page cache and lets the OS serve recent reads from RAM, avoiding GC pressure. (3) **Zero-copy** — `sendfile()` moves data from page cache to socket without copying through userspace. (4) **Batching and compression** — producers batch and compress together, giving better compression ratios and fewer, larger I/O operations.
5. When should you choose RabbitMQ or SQS over Kafka? When you need a work queue rather than an event log: a single consumer group, no replay requirement, modest volume, and especially when you need per-message acknowledgment, selective redelivery, per-message TTLs, priority queues, or complex routing (RabbitMQ's topic/header exchanges). Also whenever operational simplicity matters more than throughput — SQS requires zero operations, while Kafka needs broker sizing, partition planning, retention tuning, rebalance configuration, and lag monitoring. If the honest answer to "do we need replay or multiple independent consumers?" is no, Kafka is probably the wrong tool.

Further reading