system-design

Concurrency, Parallelism, Threads, and Async

Why one server can handle 100,000 simultaneous connections with four threads, and why adding threads eventually makes things slower.

Prerequisites: Computer Fundamentals Time to read: ~18 minutes


The problem

Your server handles one request at a time. A request takes 100 ms, of which 95 ms is waiting for the database.

Request 1: [work 5ms][──────── waiting 95ms ────────]
Request 2:                                            [work 5ms][──── waiting ────]

Throughput: 10 requests/second. Meanwhile your CPU is 95% idle, doing nothing but waiting.

The fix is obvious — handle other requests during the wait. How you do that is one of the most consequential decisions in server design, and the options have very different scaling properties.


Concurrency vs parallelism

People use these interchangeably. They’re different.

Concurrency = dealing with many things at once. A structure where multiple tasks are in progress, interleaved.

Parallelism = doing many things at once. Requires multiple CPU cores.

🧠 Rob Pike’s framing: concurrency is one barista taking orders, starting a drink, taking the next order while the espresso brews, and juggling five drinks in progress. Parallelism is hiring four baristas.

You can have concurrency without parallelism (Node.js on one core juggles thousands of connections). You can have parallelism without concurrency (a numeric computation split across 8 cores, no interleaving). Most real servers want both.

Which one you need depends on your bottleneck:

Workload Bottleneck You need
I/O-bound — waiting on database, network, disk Waiting Concurrency. More cores don’t help; you’re idle
CPU-bound — encoding video, ML inference, compression Computation Parallelism. More cores directly help

🚨 Most web services are overwhelmingly I/O-bound. This is why Node.js — single-threaded — can outperform a naively threaded server, and why “we’ll add more CPU” often changes nothing.


Model 1: thread per request

The traditional approach. Each request gets an OS thread. When it blocks on I/O, the OS parks it and runs another.

Thread 1: [work][──── blocked on DB ────][work]
Thread 2:       [work][──── blocked ────][work]
Thread 3:             [work][──── blocked ────][work]

Why it’s nice: the code is completely straightforward. Sequential, readable, easy to debug, and stack traces make sense.

def handle_request(req):
    user = db.get_user(req.user_id)      # blocks — that's fine
    orders = db.get_orders(user.id)      # blocks
    return render(user, orders)

Where it breaks:

📐 This is the C10K problem — the historical challenge of serving 10,000 concurrent connections on one machine. Thread-per-connection couldn’t do it. That constraint produced Nginx, Node.js, and every event-loop server since.

Used by: classic Java servlet containers (pre-virtual-threads), Python WSGI (Gunicorn sync workers), Ruby, PHP-FPM, Go (but see below — goroutines change the math entirely).


Model 2: event loop / async I/O

One thread. A loop. Non-blocking I/O.

Event loop:
  ├─ start DB query for req 1  → register callback, move on
  ├─ start DB query for req 2  → register callback, move on
  ├─ req 1's data arrived      → run its callback
  ├─ start DB query for req 3  → register callback, move on
  └─ req 2's data arrived      → run its callback

The thread never blocks. It asks the OS “which of these thousands of sockets is ready?” (epoll/kqueue/io_uring) and processes whichever are.

async function handleRequest(req) {
    const user = await db.getUser(req.userId);      // yields to the loop
    const orders = await db.getOrders(user.id);     // yields to the loop
    return render(user, orders);
}

Note the code looks sequential — async/await gives you readability without threads. That’s why it won.

What you gain: 100,000+ concurrent connections on one thread. Minimal memory per connection (kilobytes, not megabytes). No context-switch overhead.

What it costs:

🚨 One blocking call ruins everything. In a thread model, a slow operation blocks one thread. In an event loop, it blocks every request on that loop:

// Catastrophic in an event loop
const hash = bcrypt.hashSync(password);   // 100 ms of CPU, synchronous
// Every other request — all 5,000 of them — waits 100 ms.

The rules that follow: never do synchronous I/O, never do heavy CPU work on the loop (offload to a worker thread or a separate service), and audit every library you use for hidden blocking calls.

Also: one event loop uses one core. Run one process per core (Node’s cluster, or several container replicas) to use the whole machine.

Used by: Node.js, Nginx, Redis, Python asyncio, Netty, Rust Tokio.


Model 3: lightweight threads (green threads / coroutines)

The best of both — and where the industry is heading.

The runtime provides threads that are not OS threads: they’re cheap (a few KB), created in microseconds, and multiplexed by the runtime onto a small pool of OS threads. When one blocks, the runtime swaps in another with no kernel involvement.

// Go — looks like blocking code, scales like an event loop
func handleRequest(w http.ResponseWriter, r *http.Request) {
    user := db.GetUser(r.UserID)      // "blocks" the goroutine, not the OS thread
    orders := db.GetOrders(user.ID)
    render(w, user, orders)
}
// Go spawns a goroutine per request automatically. Millions are fine.

Why this is the sweet spot: you write simple sequential code, and get event-loop scaling. No callback complexity, no “don’t block the loop” discipline, and it uses all cores automatically.

Costs: the runtime must control the I/O layer (calling into blocking C libraries can still pin an OS thread), and stack traces / debugging can be less obvious.

Used by: Go (goroutines), Erlang/Elixir (processes), Java 21+ (virtual threads — a very big deal for the JVM ecosystem), Kotlin coroutines, Rust async tasks.


The comparison

  Thread per request Event loop Lightweight threads
Max concurrency ~1,000s ~100,000s ~1,000,000s
Memory per unit ~1 MB ~KB ~2–8 KB
Code style Simple, sequential async/await or callbacks Simple, sequential
Uses all cores ❌ (one process per core)
A blocking call Hurts one request Hurts everything Hurts one goroutine
Debugging Easy Harder Medium
Examples Classic Java, Python WSGI, PHP Node, Nginx, Redis Go, Elixir, Java 21+

The hard part: shared mutable state

Concurrency’s real difficulty isn’t scheduling. It’s that two things touching the same data at the same time produces bugs that are non-deterministic, unreproducible, and appear only under load.

Race condition — the canonical example:

Thread A: read balance (100)
Thread B: read balance (100)
Thread A: write 100 − 30 = 70
Thread B: write 100 − 50 = 50    ← A's withdrawal vanished. Balance should be 20.

This is exactly the same problem as a distributed lost update — which is why understanding it locally makes transactions and isolation levels click immediately.

The tools:

Tool What it does Watch out for
Mutex / lock Only one thread in the critical section Contention (serializes your parallelism); deadlock
Read-write lock Many readers or one writer Writer starvation
Atomic operations Lock-free single-variable updates (CAS) Only works for simple operations
Immutability Nothing to race on Allocation cost
Message passing Don’t share memory; send copies Erlang/Go channels; requires structural change
Single-threaded Sidestep it entirely Redis’s approach — no locks, one core

🧠 The best strategy is avoidance. Redis is single-threaded for its command loop and gets ~100,000 ops/sec precisely because it has no locks, no contention, and no context switches. Go’s motto — “don’t communicate by sharing memory; share memory by communicating” — is the same idea.

Deadlock — two threads each holding what the other needs, both waiting forever. Prevention: always acquire locks in a globally consistent order, and use timeouts on lock acquisition. This reappears verbatim in distributed locking, where it’s even harder because you can’t inspect the other party.

Amdahl again: every lock is a serial section. → Scalability


Practical implications for system design

Sizing thread and connection pools. Use Little’s Law (Performance Metrics):

concurrency = throughput × latency
1,000 QPS × 50 ms = 50 concurrent requests → ~50–100 pool slots

🚨 A classic production failure: 50 app servers × 100 connections each = 5,000 connections to a database that performs best at ~200. The database thrashes. Fix: smaller per-instance pools, or a connection pooler like PgBouncer in front. → Connection Pooling

Parallelize independent calls. If a request needs three unrelated services, calling them sequentially costs the sum; in parallel it costs the max.

Sequential: 30 + 40 + 50 = 120 ms
Parallel:   max(30, 40, 50) = 50 ms

This is often the single largest latency win available in a service, and interviewers look for it.

Bound your concurrency. Unbounded parallelism is a way to DDoS your own dependencies. Use a semaphore or worker pool with a fixed size. “Spawn a goroutine per item” over a million items will melt something downstream.

Separate CPU-bound work. Video transcoding, image processing, and PDF generation don’t belong in your request path. Push them to a queue and a worker fleet you can scale independently.


⚖️ Trade-offs

Choice Gain Cost
Thread-per-request Simplest code Memory and context-switch ceiling at ~1,000s
Event loop Enormous connection counts, tiny memory One blocking call stalls everything; one core per loop
Lightweight threads Simple code + high scale Runtime must own I/O; some debugging friction
Locks Correctness on shared state Contention, deadlock, serialization
Immutability / message passing No races by construction Copying cost, structural change
Larger pools More concurrency Downstream overload; more memory

In the real world


🚨 Interview traps


🎙️ Soundbites


🛠️ Try it

1. Feel the difference. Write the same “fetch from a slow API and return” endpoint twice: once blocking (Flask/sync), once async (FastAPI/Node). Load-test both with 500 concurrent connections. The gap will be dramatic, and it’ll be memory as much as throughput.

2. Block the event loop. In a Node server, add crypto.pbkdf2Sync(...) with high iterations to one endpoint. Hit a different endpoint while it runs. Watch unrelated requests stall. This is the single most important thing to internalize about async runtimes.

3. Create a race condition. Two threads incrementing a shared counter a million times. Run it — the result won’t be 2,000,000. Now add a lock. Then measure how much slower it got. That slowdown is the cost of coordination, and it’s the same cost that shows up in distributed systems.


Check yourself

1. Your service is at 20% CPU but can't handle more requests. What's likely wrong? You're I/O-bound and concurrency-limited, not CPU-limited. Likely: the thread pool or worker count is too small, or the database connection pool is exhausted so requests queue waiting for a connection, or you're using a synchronous framework with limited workers. Adding CPU or bigger instances won't help — you need more concurrent in-flight requests (larger pools, async model, or more instances). Check queue depth and pool wait time, not CPU.
2. Why is a single blocking call so much worse in Node.js than in a Java thread-per-request server? In thread-per-request, a blocking call parks *one* thread; the OS schedules others and the remaining requests proceed normally. In an event loop, all requests share one thread — so a synchronous 100 ms operation stalls every other in-flight request by up to 100 ms. A blocking call in an event loop converts a local problem into a global one.
3. When do more CPU cores actually help a web service? When the work is CPU-bound: encryption/TLS termination, compression, image or video processing, serialization of large payloads, template rendering, ML inference, and garbage collection. For a typical service that mostly waits on a database and other services, cores sit idle and the constraint is concurrency, not compute. Measure first: high CPU means add cores; low CPU with high latency means add concurrency or fix the dependency.
4. Why is Redis fast despite being single-threaded for command execution? Because its bottleneck is memory access and network I/O, not CPU. Single-threading eliminates lock acquisition, contention, and context switching entirely — every command runs to completion with no coordination overhead — and it makes every operation atomic for free. It uses an event loop for I/O multiplexing, so one thread serves tens of thousands of connections. The cost is that one Redis process uses one core, which is why you shard across multiple instances to use a whole machine.
5. You have 40 app servers, each with a 100-connection database pool. What's the problem? 4,000 potential connections to a database that typically performs best with a few hundred. Each connection costs memory and a backend process (in Postgres, ~5–10 MB each), and beyond a point the database spends its time context-switching rather than executing queries — throughput *drops* as connections rise. Fixes: shrink per-instance pools (use Little's Law to size them honestly), and put a connection pooler like PgBouncer between the app tier and the database to multiplex thousands of client connections onto a small number of real ones.

Further reading