system-design

Blob / Object Storage

Where files go. Effectively infinite, absurdly durable, remarkably cheap — and the answer to “where do we put the images?” in almost every design.

Prerequisites: CDN, Databases Overview Time to read: ~18 minutes

The problem

Users upload profile photos. Where do they go?

Option A — in the database, as a BLOB. 🚨 Almost always wrong. Your 50 GB database becomes 5 TB. Backups take hours and restores take longer. Replication ships megabytes per row. Your buffer pool fills with image bytes instead of the indexes you actually query, and every query gets slower. You’re paying database prices (~$0.10+/GB with replication) for data that needs none of a database’s features.

Option B — on the server’s local disk. 🚨 Also wrong, and it breaks the moment you have two servers. The upload lands on server 1; the next request hits server 2, which returns 404. It makes your app tier stateful, so instances aren’t interchangeable, and a disk failure loses user data permanently.

Option C — object storage. A separate system built for exactly this.

What object storage is

A flat key-value store for large immutable blobs, accessed over HTTP.

PUT  /my-bucket/users/42/avatar.jpg     (with metadata + content-type)
GET  /my-bucket/users/42/avatar.jpg
DELETE /my-bucket/users/42/avatar.jpg

That’s essentially the whole API. What you get for that simplicity:

Property	Detail
Effectively unlimited	Petabytes. No capacity planning, no provisioning.
Extreme durability	S3 advertises 99.999999999% — eleven nines. Data is erasure-coded across multiple facilities.
Cheap	~$0.023/GB/month, and far less for cold tiers. Compare to ~$0.10/GB for block storage.
HTTP-native	Serve directly to browsers; put a CDN in front trivially.
Versioning	Keep every version of an object; recover from accidental deletes.
Lifecycle rules	Automatically move to cheaper tiers or delete after N days.
Server-side encryption	At rest by default, with your keys or theirs.

🚨 Eleven nines of durability, but only ~99.99% availability. The distinction matters: your data is essentially never lost, but you might not be able to read it for a few minutes. → Availability & Reliability

Systems: AWS S3, Google Cloud Storage, Azure Blob Storage, Cloudflare R2, Backblaze B2, MinIO (self-hosted, S3-compatible).

Object storage vs block vs file

A distinction that gets asked, and the differences have real consequences.

	Block storage	File storage	Object storage
Abstraction	Raw disk	Directory tree	Flat key → blob
Access	Mount it, use a filesystem	NFS/SMB mount	HTTP API
Partial update	✅ Modify any byte	✅	❌ Replace the whole object
Attach to	One machine at a time	Many machines	Anything with HTTP
Latency	~0.1 ms	~1 ms	~10–100 ms
Scale	Limited by volume size	Limited	Effectively unlimited
Cost/GB	High (~$0.10)	High	Low (~$0.023)
Examples	EBS, local SSD	EFS, NFS, FSx	S3, GCS, Azure Blob

🚨 Objects are immutable. You cannot append a byte or edit in place — you replace the entire object. This is why object storage is wrong for database files, active logs being written, or anything requiring random writes. It’s exactly right for photos, videos, backups, and static assets.

The latency point matters too. ~50 ms for a first byte is fine for a photo behind a CDN and completely wrong for something in a request’s hot path. Don’t put your session store in S3.

The pattern: metadata in the database, bytes in object storage

This is the standard design, and it’s the answer expected in interviews.

CREATE TABLE photos (
    id           BIGSERIAL PRIMARY KEY,
    user_id      BIGINT NOT NULL,
    storage_key  TEXT NOT NULL,          -- 'photos/2026/07/a3f5c9e2.jpg'
    content_type TEXT NOT NULL,
    size_bytes   BIGINT NOT NULL,
    width        INT,
    height       INT,
    created_at   TIMESTAMPTZ DEFAULT now()
);

The database stores what you query and join on; object storage stores the bytes. The database stays small and fast; storage stays cheap and infinite.

🎙️ “Photo metadata goes in Postgres — that’s what we query and join on. The bytes go to S3 behind a CDN. Putting 2 MB blobs in the database would balloon it and push our indexes out of the buffer pool.”

Direct upload with pre-signed URLs

🚨 Never proxy large uploads through your application servers. It’s one of the most valuable things to say in a design interview involving media.

The naive flow:

Client → [2 MB] → Your API server → [2 MB] → S3

Your server’s bandwidth, memory, and request duration are all consumed moving bytes it does nothing with. At 1 Gbps, one server caps out around 60 concurrent 2 MB uploads.

The correct flow — pre-signed URLs:

sequenceDiagram
    participant C as Client
    participant A as API
    participant S as S3
    C->>A: POST /photos/upload-url (filename, size, type)
    Note over A: authorize, validate size/type,<br/>create DB row (status: pending)
    A-->>C: pre-signed PUT URL (expires in 5 min) + photo_id
    C->>S: PUT the bytes directly ⚡ (never touches your servers)
    S-->>C: 200 OK
    C->>A: POST /photos/{id}/complete
    Note over A: mark ready, enqueue thumbnail job

The pre-signed URL is a time-limited, cryptographically signed link that grants permission for one specific operation on one specific key. S3 validates the signature itself.

What your API still controls: who may upload, the key/path, the maximum size, the allowed content type, and the expiry. You keep authorization; you just don’t carry the bytes.

Downloads work the same way — issue a pre-signed GET for private content, so the CDN serves it without your origin being involved per request. → CDN

🚨 The orphan problem is the follow-up question. If the client uploads and then never calls /complete (they closed the tab), you have bytes in S3 with no database row — or a pending row forever. Fix: a lifecycle rule deleting incomplete uploads after 24 hours, plus a reconciliation job comparing storage to database rows. Mentioning this unprompted is a strong operational signal.

Storage classes and lifecycle policies

Not all data deserves the same price.

Class	Cost/GB/mo	Retrieval	Use for
Standard	~$0.023	Instant	Active content
Infrequent Access	~$0.0125	Instant, per-GB fee	Accessed monthly
Glacier Instant	~$0.004	Instant, higher fee	Archives you might need fast
Glacier Flexible	~$0.0036	Minutes to hours	Compliance archives
Deep Archive	~$0.00099	12 hours	Long-term retention you’ll probably never read

Lifecycle rules automate the transitions:

Day 0    → Standard
Day 30   → Infrequent Access
Day 90   → Glacier
Day 365  → Deep Archive
Day 2555 → Delete (7-year retention policy satisfied)

📐 Why this matters: 1 PB in Standard is ~$23,000/month. The same petabyte in Deep Archive is ~$1,000/month. For data like old user uploads or compliance logs, lifecycle policies are a 20× cost reduction for one config file.

🚨 The trap: retrieval from cold tiers costs money and time. A “cheap” archive you actually read often is more expensive than Standard. And IA/Glacier have minimum storage durations (30/90/180 days) — deleting early still bills you for the minimum. Also: transitioning has a per-object fee, so lifecycle-ing billions of tiny objects can cost more than it saves.

🎙️ “I’d put uploads in Standard and lifecycle them to Infrequent Access at 30 days and Glacier at 90 — most photos are viewed heavily in the first week and almost never after. That’s roughly a 5× storage cost reduction with no user-visible change.”

Consistency

S3 has been strongly read-after-write consistent since December 2020. Write an object, read it, and you get the new version — including overwrites and deletes.

🚨 This changed, and older material says otherwise. If you say “S3 is eventually consistent, so we need to handle stale reads,” you’ll be describing a system that hasn’t existed for years. Know the current behaviour.

What’s still true:

No object locking or transactions. Two concurrent writes to the same key: one wins, silently. Use conditional writes/ETags if that matters.
Listing is strongly consistent now too, but listing a bucket with millions of objects is slow and paginated — don’t design around it.
Cross-region replication is asynchronous and therefore eventually consistent.

Practical design notes

Key naming. Keys are flat strings; “folders” are just prefixes. Historically, sequential prefixes (2026-07-22-...) created hot partitions, and the advice was to add a random prefix. S3 now auto-scales partitions, so this is largely obsolete — but organizing keys by a hash prefix or by tenant is still good practice for listing performance and for lifecycle rules.

Multipart upload. For files over ~100 MB, upload in parallel chunks. You get better throughput, resumability (retry only the failed part), and the ability to upload files larger than 5 GB. Required over 5 GB.

Versioning. Keeps every version of an object. Excellent protection against accidental deletion or a bad deploy overwriting assets. Costs storage for every version, so pair it with a lifecycle rule that expires non-current versions.

Costs beyond storage. 🚨 Egress is usually the biggest line item. Storage at $0.023/GB looks cheap; egress at $0.05–0.09/GB does not. Serving 100 TB/month costs ~$2,300 to store and ~$7,000+ to serve. This is a major reason to put a CDN in front (cheaper egress, and it absorbs most requests), and why Cloudflare R2’s zero-egress pricing is disruptive. Request costs matter too at very high volume: millions of tiny objects incur real per-request charges.

Security. Buckets should be private by default. Public buckets are a well-known source of data breaches — misconfigured S3 buckets have exposed medical records, voter data, and credentials repeatedly. Serve private content via pre-signed URLs or a CDN with origin access control, enable “block public access” at the account level, and turn on access logging.

⚖️ Trade-offs

Decision	Gain	Cost
Object storage over DB blobs	Small fast database; cheap infinite storage	Two systems to keep consistent; orphan cleanup
Pre-signed direct upload	Servers never carry bytes	Orphan objects; more client-side complexity
Lifecycle to cold tiers	Up to 20× cheaper storage	Retrieval cost and delay; minimum durations
Versioning	Recover from mistakes	Storage cost per version
CDN in front	Lower latency and cheaper egress	Cache invalidation; another layer
Multi-region replication	Survives a region failure; lower global latency	2× storage cost; asynchronous

In the real world

Dropbox famously ran on S3, then built their own storage system (“Magic Pocket”) and moved hundreds of petabytes off it, reportedly saving substantial cost. It’s the standard example of when scale justifies leaving managed storage — and note it took them years and a dedicated team.
Netflix stores its entire encoded catalogue in S3 and pushes popular content out to Open Connect appliances. Object storage is the durable source of truth; the CDN is the serving layer. That split — durable origin, distributed cache — is the general pattern.
The 2017 S3 outage in us-east-1 broke a large fraction of the internet, including services that had assumed S3 was infallible. A useful reminder that eleven nines of durability is not eleven nines of availability, and that multi-region matters for genuinely critical assets.

🚨 Interview traps

Storing images in the database. An instant negative signal. Say metadata in the DB, bytes in object storage.
Proxying uploads through app servers. Use pre-signed URLs and say why.
Forgetting the CDN. Object storage is durable, not fast or cheap to serve globally.
Saying S3 is eventually consistent. Outdated since 2020.
Ignoring egress cost. It usually dominates storage cost.
No orphan/cleanup story for failed uploads.
Using object storage for something needing partial writes — logs being appended, database files, session state.

🎙️ Soundbites

“Bytes go to S3, metadata goes in Postgres. That keeps the database small enough that its indexes stay in memory.”
“Clients upload directly to S3 with a pre-signed URL — our API authorizes and issues the URL but never carries the bytes, so upload volume doesn’t consume app server bandwidth.”
“I’d lifecycle objects to Infrequent Access at 30 days and Glacier at 90. Most media is read heavily for a week and then almost never.”
“Egress will be our biggest storage cost, not storage itself — another reason for a CDN in front, since CDN egress is cheaper and it absorbs most of the requests.”
“For failed uploads I’d add a lifecycle rule to abort incomplete multipart uploads after a day, plus a reconciliation job against the database.”

🛠️ Try it

1. Run S3 locally. MinIO in Docker is S3-compatible and takes two minutes:

docker run -p 9000:9000 -p 9001:9001 \
  -e MINIO_ROOT_USER=admin -e MINIO_ROOT_PASSWORD=password123 \
  minio/minio server /data --console-address ":9001"

Then use the AWS SDK against http://localhost:9000. Everything you learn transfers to real S3.

2. Implement the pre-signed upload flow end to end. An endpoint that issues a pre-signed PUT, a client that uploads directly, and a completion callback. This is genuinely the most useful 45 minutes in this chapter — it’s a pattern you’ll implement repeatedly in real work, and it makes the interview answer concrete.

3. Prove the orphan problem exists. Request a pre-signed URL, upload the file, then don’t call the completion endpoint. Look at your database and your bucket. Now write the reconciliation job.

4. Compare costs honestly. Take a realistic scenario — 10 million photos at 2 MB, each viewed 20 times a month — and compute storage vs egress vs CDN-fronted egress. The ratio will change how you argue for a CDN.

Check yourself

1. Why shouldn't you store images in your relational database?

Several compounding reasons: the database grows enormously, making backups and restores slow and risky; replication ships megabytes per row; blob data competes with indexes for the buffer pool, so *every* query gets slower as the cache hit rate falls; you pay database-tier storage prices (~4-5× object storage) for data that needs no relational features; and you can't serve it directly to browsers or put a CDN in front without proxying it through your application. Metadata in the database, bytes in object storage.

2. What is a pre-signed URL and what problem does it solve?

A time-limited, cryptographically signed URL granting permission for one specific operation (PUT or GET) on one specific object key. Your API authorizes the user and issues the URL; the client then talks directly to the storage service, which validates the signature itself. It solves the problem of upload/download bytes flowing through your application servers — consuming bandwidth, memory, and request duration for data your code never inspects. You keep authorization control while the bytes take a direct path.

3. What's the difference between S3's durability and availability figures?

**Durability (99.999999999%)** is the probability your data still exists — it's replicated and erasure-coded across multiple facilities, so loss is essentially impossible. **Availability (~99.99%)** is the probability you can *access* it right now — roughly 52 minutes of unavailability per year. Your data can be perfectly safe and temporarily unreachable. If your system can't tolerate that window, you need multi-region replication or a CDN cache that can serve during an origin outage.

4. A user starts an upload and abandons it. What's left behind and how do you handle it?

Depending on where they abandoned: a `pending` row in your database with no object, an object in storage with no completed row, or an incomplete multipart upload consuming storage you're billed for but can't see in a normal listing. Handling: set a lifecycle rule to abort incomplete multipart uploads after 24 hours; expire `pending` database rows on a schedule; and run a periodic reconciliation job comparing object keys against database rows in both directions, deleting orphans on each side. Also make the completion step idempotent so a retried callback is harmless.

5. When is object storage the wrong choice?

When you need partial or random writes — objects are immutable, so appending a log line or updating a byte means rewriting the whole object. When you need low latency in a request's hot path (~10–100 ms first-byte is too slow for a session store or a cache). When you need a real filesystem interface for software that expects one (use EFS/NFS). When you need transactions or locking across objects. And when objects are tiny and numerous — per-request costs and per-object overhead dominate, and a database or a packed format is cheaper.