Capping how fast clients can hit you. It’s how you survive abuse, protect downstream systems, and keep one customer from starving everyone else.
Prerequisites: HTTP & TLS, Caching Time to read: ~24 minutes
Your API is public. Then one of these happens:
Without limits, the loudest client determines everyone’s experience. Rate limiting turns “the system falls over” into “that one client gets 429s.”
It’s not only about attacks. Rate limiting is fundamentally about fairness and capacity protection — a much better framing in an interview than “to stop hackers.”
Five of them, and the differences genuinely matter.
Count requests per fixed time window.
key = f"rate:{user_id}:{int(time.time() // 60)}" # per-minute bucket
count = redis.incr(key)
if count == 1:
redis.expire(key, 60)
if count > LIMIT:
return 429
✅ Trivially simple, memory-efficient (one counter per user), fast.
❌ 🚨 The boundary burst problem — and this is the classic interview question:
Limit: 100 requests/minute
10:00:59 → 100 requests (window 10:00 — allowed)
10:01:00 → 100 requests (window 10:01 — allowed)
─────────────────────────────
200 requests in 2 seconds, with a "100/minute" limit.
You can get 2× your intended rate across a window boundary. Fine for coarse quotas; bad when the limit exists to protect a fragile downstream.
Store a timestamp for every request; count those within the last N seconds.
now = time.time()
key = f"rate:{user_id}"
pipe = redis.pipeline()
pipe.zremrangebyscore(key, 0, now - WINDOW) # drop expired entries
pipe.zadd(key, {str(uuid4()): now})
pipe.zcard(key) # how many remain
pipe.expire(key, WINDOW)
_, _, count, _ = pipe.execute()
if count > LIMIT:
return 429
✅ Perfectly accurate. No boundary problem at all.
❌ Memory-expensive. You store one entry per request. A limit of 10,000/hour means up to 10,000 timestamps per user in memory. At a million users, that’s not viable.
Use for: low limits where precision matters — login attempts, password resets, expensive operations.
Approximate the sliding window using the current and previous fixed windows, weighted:
count = current_window_count
+ previous_window_count × (fraction of the previous window still in view)
# 30 seconds into the current minute
# previous window had 80 requests, current has 40
estimated = 40 + 80 × 0.5 = 80
✅ Nearly as accurate as the log, with the memory of a fixed window (two counters per user). ✅ Smooths out the boundary burst. ❌ Slightly approximate — it assumes requests in the previous window were uniformly distributed.
🎙️ This is usually the right answer in an interview: “I’d use a sliding window counter — it avoids the fixed-window boundary burst without storing a timestamp per request.”
A bucket holds tokens. Tokens refill at a constant rate. Each request consumes one. Empty bucket → rejected.
def allow(user_id, cost=1):
now = time.time()
tokens, last = redis.hmget(key, "tokens", "ts")
tokens = float(tokens or CAPACITY)
last = float(last or now)
tokens = min(CAPACITY, tokens + (now - last) * REFILL_RATE) # refill
if tokens < cost:
return False
redis.hset(key, mapping={"tokens": tokens - cost, "ts": now})
return True
🧠 Mental model: a bucket under a slow tap. It fills at a steady rate up to a maximum. You can scoop out a full bucket at once (a burst), but then you must wait for it to refill.
✅ Allows bursts while enforcing a long-run average. This matches real usage: a client that’s been idle for ten minutes has “saved up” credit and can legitimately make a batch of requests. ✅ Memory-efficient — two values per user. ✅ Supports weighted costs. An expensive endpoint can cost 10 tokens; a cheap one, 1. This is how you rate limit fairly across heterogeneous operations, and it’s a nice detail to mention.
This is what most production API rate limiters actually use — Stripe, GitHub, AWS.
Requests enter a queue and are processed at a fixed rate. Overflow is dropped.
✅ Perfectly smooth output rate — the downstream sees a constant load, never a spike. ❌ No bursts allowed, and queuing adds latency.
Use when protecting something that genuinely cannot handle bursts — a legacy system, a third-party API with a hard rate limit, or a payment processor.
⚖️ Token bucket vs leaky bucket in one line: token bucket smooths the average and permits bursts; leaky bucket smooths the instantaneous rate and forbids them.
| Algorithm | Memory/user | Accuracy | Bursts | Use when |
|---|---|---|---|---|
| Fixed window | 1 counter | Poor (2× at boundary) | Accidental | Coarse quotas, simplicity matters |
| Sliding log | N timestamps | Perfect | No | Low limits, high precision (logins) |
| Sliding counter | 2 counters | Very good | No | General-purpose default |
| Token bucket | 2 values | Good | Yes, controlled | Public APIs |
| Leaky bucket | Queue | Good | No | Protecting fragile downstreams |
The dimension matters as much as the algorithm.
| Key | Good for | Weakness |
|---|---|---|
| API key / user ID | Authenticated APIs — the best option | Requires authentication |
| IP address | Unauthenticated endpoints | 🚨 NAT/CGNAT means a whole university or country shares one IP. Attackers rotate IPs cheaply. |
| IP + endpoint | Login endpoints | Same NAT problem |
| Session / device ID | Web apps | Clients can discard them |
| Global | Protecting a specific downstream | Doesn’t isolate abusers |
🚨 The IP problem is worth raising. Rate limiting logins by IP alone means one office’s shared IP gets locked out by one careless user, while an attacker with a botnet or a rotating proxy pool is barely inconvenienced. Layer the limits: per-account (5 failed logins), per-IP (100/hour), and global (to detect a distributed attack), and require CAPTCHA or step-up auth rather than only blocking.
Layered limits are the general answer:
Per user: 1,000 requests/hour
Per user per endpoint: 10 password-resets/hour
Per IP: 5,000 requests/hour
Global: 100,000 requests/second (capacity protection)
You have 50 API servers. A user is limited to 100 requests/minute. Each server sees only its own traffic.
Option A — local counters. Each server allows 100/50 = 2 per minute. ❌ Broken: load balancers don’t distribute perfectly, so a user routed to one server hits the limit while capacity sits unused elsewhere.
Option B — centralized store (Redis). All servers increment the same counter. ✅ Accurate. ❌ A network round trip (~0.5 ms) on every request, and Redis becomes a critical dependency.
Option C — local counters with async sync. Count locally, share totals every few hundred milliseconds. ✅ No latency cost. ❌ Approximate — a client can briefly exceed the limit during the sync window.
In practice: Redis with atomic operations is the standard answer. Use a Lua script so the check-and-increment is atomic:
-- Without this, two servers can both read "99", both allow, and you're at 101.
local current = redis.call('INCR', KEYS[1])
if current == 1 then redis.call('EXPIRE', KEYS[1], ARGV[2]) end
if current > tonumber(ARGV[1]) then return 0 end
return 1
🚨 The read-then-write race is a real bug. GET then INCR as separate commands allows two
concurrent requests to both see 99 and both proceed. Use INCR (which is atomic and returns the new
value) or a Lua script.
🎙️ “I’d use Redis with a Lua script so the check and increment are atomic — otherwise concurrent requests across servers can both pass the check. It’s a 0.5 ms round trip, which is acceptable at the gateway.”
And: what happens if Redis is down? ⚖️ Fail open or fail closed?
For most public APIs, fail open with a conservative local fallback limit is right — a rate limiter should not be able to take down your service. For security-critical limits (login attempts), fail closed. Being asked this is common; having a considered answer is what matters.
flowchart LR
C[Client] --> E[CDN / Edge<br/>volumetric DDoS]
E --> G[API Gateway<br/>per-key quotas]
G --> S[Service<br/>per-operation limits]
S --> D[(Downstream<br/>client-side limiting)]
🚨 Reject as early and as cheaply as possible. A rate limiter that rejects after your service has authenticated the user, queried the database, and started rendering has already spent the resources it was meant to save.
HTTP/1.1 429 Too Many Requests
Retry-After: 30
X-RateLimit-Limit: 1000
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1735689600
Content-Type: application/json
{"error": "rate_limit_exceeded",
"message": "Rate limit exceeded. Retry after 30 seconds.",
"retry_after": 30}
🚨 Retry-After is not optional. Without it, clients retry immediately, which makes the overload
worse — you’ve built a rate limiter that amplifies the problem it’s solving.
Expose the limit headers proactively (X-RateLimit-Remaining) so well-behaved clients can
self-throttle before being rejected. GitHub does this, and it meaningfully reduces the number of
clients that hit the wall.
429 vs 503: 429 means “you specifically are sending too much.” 503 means “the service is overloaded in general.” Different causes, different client behaviour, different alerting. Don’t conflate them.
Related, distinct, and worth distinguishing precisely:
🎙️ You want both rate limiting and load shedding. Rate limiting handles fairness and abuse; load shedding handles the case where legitimate aggregate traffic exceeds capacity. Saying this distinguishes you — most candidates conflate them.
Priority-based shedding is a good detail: under load, drop analytics and prefetch requests before dropping checkout requests. Not all traffic is equally valuable.
| Decision | Gain | Cost |
|---|---|---|
| Fixed window | Simplest, cheapest | 2× burst at boundaries |
| Sliding log | Perfect accuracy | Memory proportional to request count |
| Token bucket | Allows legitimate bursts | Slightly more state; burst can hit downstream |
| Centralized (Redis) | Accurate across the fleet | Round trip per request; Redis becomes critical |
| Local counters | Zero latency | Inaccurate, unfair distribution |
| Fail open | Rate limiter can’t cause an outage | Unprotected during a Redis outage |
| Limit by IP | Works unauthenticated | NAT punishes innocents; attackers rotate IPs |
X-RateLimit-* headers on every response and doesn’t count 304 Not
Modified against your limit — a direct incentive for clients to use conditional requests.
→ HTTPRetry-After. Without it, clients retry immediately and amplify the overload.GET then SET isn’t atomic.1. Build all four in Redis. Implement fixed window, sliding log, sliding counter, and token bucket against a local Redis. Then hammer each with a script sending 200 requests in 2 seconds across a window boundary and record how many each allowed. The fixed window will let through ~2× the limit — seeing that number yourself makes the interview answer stick.
2. Prove the race condition. Implement GET → check → INCR as three separate commands. Run 100
concurrent requests against a limit of 50. Count how many got through. It’ll be more than 50. Then
convert to a Lua script and watch it become exactly 50.
3. Feel the token bucket. Set capacity 10, refill 1/second. Send 10 requests instantly (all allowed), then 1 more (rejected), then wait 5 seconds and send 5 (allowed). That behaviour — saved-up burst credit — is exactly why it suits public APIs.
4. Try it in Nginx, which has this built in:
limit_req_zone $binary_remote_addr zone=api:10m rate=10r/s;
server {
location /api/ {
limit_req zone=api burst=20 nodelay;
limit_req_status 429;
}
}
Experiment with burst and nodelay — they’re the leaky-vs-token-bucket distinction in
configuration form.