system-design

Virtual Machines, Containers, and Docker

“It works on my machine” was one of the most expensive sentences in software. Containers made it stop being true — by shipping the environment along with the code.

Prerequisites: Computer Fundamentals Time to read: ~18 minutes

The problem

You write code that works on your laptop. It fails on the server. Why? A different OS version, a missing library, a different Python version, a config file that isn’t there, an environment variable that’s set differently.

🚨 The root cause: your code doesn’t run in isolation — it depends on its entire environment, and that environment differs everywhere. Every deployment target is a slightly different snowflake, and the differences cause failures that are maddening to debug because they only happen “over there.”

The evolution of solutions: bare metal → virtual machines → containers, each packaging more of the environment with the code.

The progression

Bare metal

One OS, one machine, your app directly on it.

❌ Slow to provision (physical hardware), poor utilization (one app can’t fill a machine, but two apps conflict), no isolation, hard to reproduce.

Virtual machines

A hypervisor runs multiple complete virtual computers on one physical machine, each with its own full OS.

┌─────────────────────────────────────┐
│  App A  │  App B  │  App C           │
│ Bins/Lib│ Bins/Lib│ Bins/Lib         │
│ Guest OS│ Guest OS│ Guest OS  ← each a FULL OS (GBs)
├─────────┴─────────┴──────────────────┤
│            Hypervisor                │
│            Host OS + Hardware        │
└─────────────────────────────────────┘

✅ Strong isolation (each VM is a full separate machine), run different OSes, mature. ❌ 🚨 Heavy — each VM includes a full OS (gigabytes, minutes to boot), so you can fit few per host and utilization is poor. Duplicating a full OS per app is wasteful.

Containers

🚨 The key idea: containers share the host’s OS kernel, isolating only the process and its dependencies — not a whole OS.

┌─────────────────────────────────────┐
│  App A  │  App B  │  App C           │
│ Bins/Lib│ Bins/Lib│ Bins/Lib  ← just the app + its deps (MBs)
├─────────┴─────────┴──────────────────┤
│         Container runtime (Docker)   │
│         Shared Host OS Kernel        │  ← ONE kernel, shared
│              Hardware                │
└─────────────────────────────────────┘

✅ 🚨 Lightweight — megabytes not gigabytes, starts in milliseconds not minutes, so you fit many per host with high utilization. ✅ Consistent — the container bundles the app and its dependencies, so it runs identically everywhere. “Works on my machine” becomes “works in the container, which runs the same everywhere.” ✅ Fast to build, ship, and start — enabling rapid deploys and scaling.

❌ Weaker isolation than VMs — they share the kernel, so a kernel exploit escapes the container; a security concern for running untrusted code (multi-tenant, someone else’s code). ❌ Same-kernel constraint — Linux containers need a Linux kernel (Docker on Mac/Windows runs a hidden Linux VM).

How containers actually work

🚨 Containers aren’t magic — they’re Linux kernel features, and knowing this dispels the mystery:

Namespaces — isolate what a process can see: its own process tree (PID namespace), network stack, mounts, hostname, users. The process thinks it’s alone on the machine.
cgroups (control groups) — limit what a process can use: CPU, memory, I/O. This is how you cap a container at “1 CPU, 512MB.”
Union filesystems (overlayfs) — layered, copy-on-write filesystems, which is what makes images efficient (shared layers).

A container is just a normal Linux process with namespaces restricting its view and cgroups limiting its resources. That’s the whole trick — it’s not a lightweight VM, it’s an isolated process.

Images and layers

An image is the packaged, immutable template; a container is a running instance of an image. (Image is to container as class is to object.)

Images are built in layers, each a filesystem change:

FROM python:3.12-slim          # base layer (shared across all Python images)
WORKDIR /app
COPY requirements.txt .        # layer
RUN pip install -r requirements.txt   # layer (the expensive one)
COPY . .                       # layer (changes most often)
CMD ["python", "app.py"]

🚨 Layers are cached and shared, which has two big consequences:

Build caching — if a layer’s inputs haven’t changed, it’s reused. This is why 🚨 you order Dockerfile instructions least-changing to most-changing: copy requirements.txt and install deps before copying your code, so a code change doesn’t re-run the slow dependency install. Getting this order wrong makes every build slow.
Storage/transfer efficiency — many images share base layers, so you store and transfer them once.

Image size matters:

🚨 Use minimal base images (slim, alpine, or distroless) — a smaller image is faster to pull, deploy, and scale, and has a smaller attack surface.
Multi-stage builds — build in a heavy image with all the build tools, then copy only the artifact into a tiny runtime image. The final image doesn’t carry the compiler and build dependencies.

Container best practices (that show up in interviews)

🚨 These come up, and getting them right signals real experience:

One process per container — a container should do one thing (a web server, or a worker), so it scales, logs, and fails as a unit. Not a whole app stack in one container.
Stateless containers — 🚨 containers are ephemeral; they can be killed and recreated anytime. Never store persistent state inside — data goes to a database or volume, files to object storage, sessions to Redis. This is the statelessness principle at the container level, and it’s what makes containers freely disposable.
Config from the environment, secrets injected at runtime — 🚨 never baked into the image (image layers are inspectable and shared). → Secrets Management
Don’t run as root — a container running as root that’s compromised has more privilege to escape.
Health check endpoints so the orchestrator knows if the container is alive/ready. → Load Balancers
Handle graceful shutdown (SIGTERM) — finish in-flight requests before exiting, so deploys don’t drop requests.

Why containers changed everything

🚨 Containers are the substrate that enabled the modern deployment world, and the connections are worth naming:

Microservices became practical — packaging and deploying dozens of independent services is only manageable when each is a lightweight, consistent container.
CI/CD — build the image once, deploy the exact same artifact everywhere (dev, staging, prod), eliminating environment drift between stages.
Kubernetes and orchestration — containers are the unit orchestrators schedule and scale.
Immutable infrastructure — instead of modifying running servers, you build a new image and replace the container. → Deployment Strategies
Serverless containers (Cloud Run, Fargate).

The through-line: containers made the deployment artifact portable and identical everywhere, which is the foundation everything else builds on.

When VMs still make sense

⚖️ Containers aren’t always the answer:

Stronger isolation needed — running untrusted code, hard multi-tenancy, or strict compliance where kernel sharing is unacceptable. (Though microVMs like Firecracker — used by AWS Lambda and Fargate — give VM-level isolation with near-container speed, a modern middle ground worth knowing.)
Different OS or kernel required.
Legacy apps not designed for containers.
Full-machine workloads — a database that wants a whole tuned machine may run fine on a VM or bare metal.

Often the answer is both: VMs for isolation boundaries, containers packed inside them (which is exactly how most managed Kubernetes runs — containers inside VMs).

⚖️ Trade-offs

	VMs	Containers
Isolation	Strong (full OS)	Weaker (shared kernel)
Size	GBs	MBs
Startup	Minutes	Milliseconds
Density per host	Low	High
OS flexibility	Any OS	Host’s kernel only
Portability	Heavy images	Lightweight, consistent
Best for	Isolation, legacy, different OS	Microservices, CI/CD, rapid scaling

In the real world

Docker (2013) didn’t invent containers (Linux namespaces/cgroups and LXC predate it) — it made them usable, with a simple build format (Dockerfile), an image registry, and tooling. That usability triggered the container revolution and everything that followed.
Firecracker (AWS) is the microVM that powers Lambda and Fargate — VM-level isolation with ~125ms startup, purpose-built to run untrusted customer code safely at serverless density. It’s the answer to “containers are too weakly isolated for multi-tenant untrusted code.”
The “works on my machine” problem genuinely largely disappeared with containers — the single most impactful practical benefit, because the environment ships with the code and the artifact built in CI is the exact one that runs in production.

🚨 Interview traps

Confusing VMs and containers — containers share the kernel; VMs each have a full OS.
Not knowing containers are just processes with namespaces + cgroups.
Storing state inside containers — they’re ephemeral.
Baking secrets into images.
Wrong Dockerfile layer ordering making builds slow.
Running multiple processes / a whole stack in one container.
Ignoring the weaker isolation for untrusted/multi-tenant workloads.

🎙️ Soundbites

“Containers share the host kernel and isolate only the process and its dependencies, so they’re megabytes and start in milliseconds — versus VMs, which each carry a full OS and boot in minutes. That density and the bundled dependencies are why ‘works on my machine’ stopped being a problem.”
“A container is really just a Linux process with namespaces restricting what it sees and cgroups limiting what it uses. It’s not a lightweight VM, it’s an isolated process.”
“Containers are ephemeral, so nothing persistent lives inside — state goes to a database or volume, sessions to Redis, files to object storage. That’s what makes them freely disposable.”
“I’d order the Dockerfile least-changing to most-changing — install dependencies before copying code — so a code change doesn’t invalidate the cache and re-run the slow install. And a minimal base image with a multi-stage build to keep it small.”
“For untrusted multi-tenant code, containers’ shared kernel is a concern — I’d use microVMs like Firecracker, which give VM-level isolation at near-container speed. That’s what Lambda and Fargate do.”

🛠️ Try it

1. Containerize an app. Write a Dockerfile for a small app, build the image, run it. Then run it on a different machine (or give the image to a colleague) and confirm it behaves identically. That portability — the exact same artifact running anywhere — is the whole point, felt directly.

2. Prove layer caching. Build an image, change a line of code, rebuild — note it re-runs the dependency install (if your ordering is wrong). Reorder so deps install before code copy, and rebuild after a code change — watch it skip straight to the fast layer. The build-time difference is dramatic.

3. See namespaces and cgroups. Run a container and inspect it: docker exec in and note the process sees PID 1 as its own app (PID namespace), a private network, and its own filesystem. Then docker run --memory=100m and watch a memory-hungry process get killed at the cgroup limit.

4. Shrink an image. Take an image built from a full base and rebuild it with a slim/alpine base and a multi-stage build. Compare sizes — often a 10× reduction, which directly speeds pulls and scaling.

Check yourself

1. What's the fundamental difference between a VM and a container?

A VM virtualizes an entire machine: a hypervisor runs multiple guests, *each with its own complete operating system* including its own kernel, on top of the host. A container shares the host's OS kernel and isolates only the process and its dependencies — there's one kernel, and containers are isolated processes running on it. This is why containers are megabytes and start in milliseconds (they're not booting an OS, just starting a process), while VMs are gigabytes and boot in minutes (each carries a full OS). The trade-off is isolation: VMs are fully separate machines with strong isolation, while containers share a kernel, so a kernel-level exploit can escape a container in a way it couldn't escape a VM — which matters for running untrusted code. Containers give density and speed; VMs give stronger isolation.

2. What Linux features make containers work, and why does that matter?

Three kernel features. **Namespaces** isolate what a process can *see* — its own process tree (so it sees itself as PID 1, alone on the machine), its own network stack, filesystem mounts, hostname, and users. **cgroups** (control groups) limit what a process can *use* — capping its CPU, memory, and I/O (this is how you enforce "512MB, 1 CPU" per container). **Union/overlay filesystems** provide layered, copy-on-write storage, making images efficient through shared layers. It matters because it demystifies containers: a container is not a lightweight VM or anything magical — it's an ordinary Linux process with namespaces restricting its view and cgroups bounding its resource use. Understanding this explains their properties (why they're fast, why they share the kernel, why isolation is weaker than a VM) and why they need a Linux kernel to run.

3. Why should containers be stateless, and where does state go?

Because containers are ephemeral — the orchestrator can kill, move, and recreate them at any time (for scaling, rescheduling, deploys, or node failure), and anything stored inside a container is lost when it's replaced. Depending on state inside a container breaks freely disposing of them, which is the whole operational benefit. So state goes elsewhere: application data to a database, persistent files to object storage or a mounted volume, session state to a shared store like Redis, and cached data to a distributed cache. This is the same statelessness principle that enables horizontal scaling, applied at the container level — with state externalized, any container instance is interchangeable, so you can add, remove, restart, and reschedule them without user impact. A stateful container is a container you can't safely move, which defeats orchestration.

4. Why does Dockerfile instruction order affect build speed?

Because each instruction creates a cached layer, and Docker reuses a layer only if that instruction and all the layers before it are unchanged. When any layer changes, that layer and every layer after it must rebuild. So you order instructions from least-frequently-changing to most-frequently-changing: copy the dependency manifest (`requirements.txt`, `package.json`) and install dependencies *before* copying your application code. Dependencies change rarely, so their expensive install layer stays cached across builds; application code changes constantly, but since it's copied last, a code change only invalidates the fast final layers, not the slow dependency install. Get the order wrong — copy all code first, then install — and every single code change re-runs the full dependency installation, making every build slow. Correct ordering can turn a multi-minute build into a few seconds.

5. When are VMs (or microVMs) still the right choice over containers?

When you need stronger isolation than a shared kernel provides. The main cases: **running untrusted code** — multi-tenant platforms executing customers' arbitrary code can't safely rely on containers, because a kernel exploit escapes the container into the host and potentially other tenants. **Strict compliance or security requirements** where kernel sharing is unacceptable. **A different OS or kernel version** than the host. **Legacy applications** not designed for containers. And **full-machine workloads** like a heavily-tuned database that wants dedicated hardware. The modern middle ground is **microVMs** (Firecracker), which give VM-level isolation with container-like startup (~125ms) and density — which is exactly why AWS Lambda and Fargate use them to run untrusted customer code safely. Often the real answer is both: containers packed inside VMs, which is how most managed Kubernetes actually runs — the VM provides the isolation boundary, containers provide the density within it.