system-design

Infrastructure as Code

Your servers, networks, and databases defined in files you commit to git — not clicked together by hand in a console. Because “how is production configured?” should have an answer, not a shrug.

Prerequisites: Containers, CI/CD Time to read: ~14 minutes

The problem

Someone set up your production infrastructure by clicking around a cloud console — creating servers, configuring networks, setting security groups. Now:

🚨 Nobody knows exactly how it’s configured. The knowledge lives in one person’s memory (and they left).
You can’t reproduce it. Setting up a staging environment that matches production is guesswork.
Changes aren’t tracked. Who changed the firewall rule, when, and why? No record.
It drifts. Manual tweaks over time mean the actual state diverges from any documentation.
Disaster recovery is a nightmare. Rebuilding after a region failure means re-clicking everything from memory.

Infrastructure as Code (IaC) fixes all of this: define infrastructure in declarative files, stored in version control, applied by tooling. Your infrastructure becomes reproducible, reviewable, and auditable — like your application code.

The core idea: declarative infrastructure

🚨 Same principle as Kubernetes: you declare the desired state, and the tool makes reality match.

# Terraform — declares WHAT you want, not the steps to create it
resource "aws_instance" "web" {
  count         = 3
  instance_type = "t3.medium"
  ami           = "ami-0abc123"
  tags = { Name = "web-server" }
}

You don’t script “create instance, then configure it, then…” — you declare “I want 3 web servers of this type,” and the tool figures out what to create, change, or destroy to match. Run it again with the same config and nothing changes (idempotent). Change the count to 5 and it creates 2 more.

Declarative vs imperative is the key distinction: declarative (Terraform, CloudFormation) describes the end state; imperative (a bash script) describes the steps. Declarative is idempotent and self-reconciling; imperative isn’t.

What IaC gives you

🚨 These benefits are why IaC is standard, and they’re worth being able to list:

Reproducibility — spin up an identical environment (staging, DR, a new region) from the same files. No more “staging doesn’t match production.”
Version control — infrastructure changes go through git: reviewed in pull requests, with full history of who changed what and why. → CI/CD
Auditability — every change is a tracked commit.
Consistency — no manual drift; the code is the source of truth.
Disaster recovery — rebuild everything from code, fast.
Self-documenting — the code is the documentation of your infrastructure (and it’s always current, unlike a wiki).
Testable / reviewable — see the plan (what will change) before applying.

🚨 The security angle (from OWASP misconfiguration): IaC makes configuration reviewable, so a misconfigured security group or public bucket gets caught in code review — versus a console click nobody sees. This is a major security benefit.

The tools

Tool	Character
Terraform / OpenTofu	Cloud-agnostic, declarative, the de-facto standard. HCL language.
CloudFormation	AWS-native, declarative. Deep AWS integration, AWS-only.
Pulumi	IaC in real programming languages (Python, TypeScript) rather than a DSL.
CDK	AWS’s “code that generates CloudFormation.”
Ansible	Configuration management (imperative-ish); provisioning + config.
Kubernetes manifests / Helm	IaC for what runs inside the cluster.

Terraform is the common answer for cloud infrastructure — cloud-agnostic and widely adopted. Knowing the concept (declarative, version-controlled infrastructure) matters more than any specific tool for interviews.

Key concepts

State — 🚨 Terraform tracks what it has created in a state file (mapping config to real resources). This state is critical: it must be stored remotely (S3, not a laptop), locked (so two people don’t apply simultaneously and corrupt it), and protected (it can contain secrets). Losing or corrupting state is a serious operational problem.

Plan / apply — 🚨 you run plan to preview what will change (create/modify/destroy) before apply actually does it. Always review the plan — it prevents accidentally destroying production (a plan showing “will destroy database” is your last chance to catch a mistake).

Modules — reusable, parameterized infrastructure components (a “web service” module used across environments), so you don’t copy-paste.

Drift — 🚨 when the real infrastructure diverges from the code (someone made a manual change). IaC tools can detect drift (the real state differs from declared) and correct it. The rule: never make manual changes — always go through the code, or drift accumulates and you lose the guarantees.

The GitOps extension

🚨 A modern practice worth knowing: GitOps applies IaC principles operationally — git is the single source of truth for both infrastructure and deployments, and an automated agent continuously reconciles the actual cluster state to match git.

git repo (desired state) → agent (Argo CD / Flux) watches → makes the cluster match

Change infrastructure or deployments by committing to git (reviewed via PR); the agent applies it automatically, and detects/reverts drift. It’s the Kubernetes reconciliation model extended to your whole delivery process — declarative, version-controlled, auto-reconciled. Mentioning GitOps (Argo CD, Flux) is a good currency signal.

Pitfalls

⚖️ IaC isn’t free of problems:

State management complexity — the state file is a critical, fragile artifact (locking, storage, secrets).
The learning curve and the “everything is now code” overhead for small setups.
Blast radius — 🚨 a bad apply can destroy production at scale (a wrong config change applied to everything at once). Review plans, use staged rollouts for infra too, and separate environments.
Secrets in state / code — IaC files and state can leak secrets; keep secrets in a secrets manager, referenced not embedded.
Drift from manual changes — the discipline of “never touch the console” is hard to enforce but essential.

For a tiny project, clicking in the console is faster; IaC’s value grows with scale, team size, and the need for reproducibility and DR.

⚖️ Trade-offs

	Gain	Cost
IaC	Reproducible, version-controlled, auditable, DR-ready, self-documenting	Learning curve; state management; overhead for tiny setups
Declarative (Terraform)	Idempotent, self-reconciling, plan preview	Less flexible than code for complex logic
GitOps	Git as source of truth, auto-reconciliation, drift correction	Requires the agent and discipline
Modules	Reuse, consistency	Abstraction to maintain

In the real world

Terraform became the de-facto standard because it’s cloud-agnostic — one tool and language for AWS, GCP, Azure, and hundreds of providers — and made “infrastructure in version control” the normal expectation rather than a novelty.
GitOps (Argo CD, Flux) has become the standard way to manage Kubernetes deployments at scale, extending the declarative-reconciliation model to the whole delivery process — you deploy by merging a PR, and the cluster converges automatically.
The “we don’t know how production is configured” problem is a real and common source of outages and slow DR — organizations that manually clicked their infrastructure together consistently struggle to reproduce, audit, or recover it, which is the strongest argument for IaC.

🚨 Interview traps

Not mentioning IaC when discussing how infrastructure is managed or reproduced.
Not knowing it’s declarative (desired state) like Kubernetes.
Ignoring state management — the state file is critical and fragile.
Not reviewing plans before apply — how you avoid destroying production.
Allowing manual changes — causes drift, loses the guarantees.
Secrets embedded in IaC/state.
Not knowing GitOps as the modern operational extension.

🎙️ Soundbites

“Infrastructure as code means the servers, networks, and databases are declared in version- controlled files, not clicked together in a console. It makes everything reproducible, reviewable, and auditable — and ‘how is production configured?’ has an answer instead of a shrug.”
“It’s declarative like Kubernetes — you declare desired state and the tool reconciles reality to match, idempotently. Change the count from 3 to 5 and it creates 2 more.”
“The security benefit is real: a misconfigured security group or public bucket gets caught in code review, versus a console click nobody sees.”
“State management is the critical part — the state file maps config to real resources, so it must be stored remotely, locked, and protected. And always review the plan before apply, so ‘will destroy database’ is caught.”
“GitOps extends this operationally — git is the source of truth and an agent continuously reconciles the cluster to match, correcting drift. It’s the reconciliation model applied to the whole delivery process.”

🛠️ Try it

1. Provision infrastructure from code. Write a small Terraform config (a VM, a security group, maybe a bucket), run plan to preview, then apply. Then delete it all with destroy and recreate it identically from the same file — that reproducibility is the whole point.

2. See the plan catch a mistake. Change your config to accidentally reduce a database’s size (or delete it). Run plan and watch it show “will destroy.” That preview is your last chance to catch a destructive change — feel why reviewing plans matters.

3. Cause and detect drift. Apply your config, then manually change something in the console (a tag, a rule). Run plan again and watch Terraform detect the drift (the real state differs from the code). This makes the “never touch the console” rule concrete.

4. Reproduce an environment. Parameterize your config into a module, then instantiate it twice (staging and prod) from the same code. Watch two identical environments come up from one definition — that’s how you stop “staging doesn’t match production.”

Check yourself

1. What problems does Infrastructure as Code solve?

The problems of manually-managed infrastructure. When infrastructure is set up by clicking in a console: nobody knows exactly how it's configured (the knowledge is in one person's head), you can't reliably reproduce it (setting up a matching staging or DR environment is guesswork), changes aren't tracked (no record of who changed what or why), it drifts over time as people make manual tweaks, and disaster recovery means re-clicking everything from memory. IaC defines infrastructure in declarative files stored in version control and applied by tooling, which makes it reproducible (spin up identical environments from the same files), version-controlled (changes reviewed in PRs with full history), auditable (every change is a commit), consistent (the code is the source of truth, no manual drift), DR-ready (rebuild from code fast), and self-documenting (the code *is* the always-current documentation). It also improves security by making misconfigurations reviewable in code review rather than invisible console clicks. Infrastructure becomes as manageable as application code.

2. What does "declarative" mean for IaC, and why does it matter?

Declarative means you describe the *desired end state* — "I want 3 web servers of this type with these tags" — rather than the *steps* to achieve it (imperative: "create a server, then configure it, then create another..."). It matters for two reasons. First, **idempotence**: running the same declarative config repeatedly produces the same result — if the 3 servers already exist, nothing changes; if you change the count to 5, it creates exactly 2 more. An imperative script run twice might create 6 servers. Second, **self-reconciliation**: the tool compares the declared state to the actual state and computes the minimal set of changes (create/modify/destroy) to close the gap, so it handles partial states, drift, and changes automatically. This is the same model as Kubernetes — declare what you want, let the tool reconcile — and it's what makes IaC safe to run repeatedly and able to detect and correct drift, versus imperative scripts that assume a specific starting state and break if reality differs.

3. Why is the Terraform state file critical, and how do you manage it?

The state file is Terraform's record of what it has actually created — a mapping between your config and the real cloud resources (their IDs, current attributes). It's critical because Terraform uses it to know what exists and what to change: without accurate state, Terraform can't tell whether to create, modify, or destroy a resource, and a corrupted or lost state file can cause it to try recreating resources that already exist or lose track of resources it should manage (orphaning them or destroying the wrong things). Management essentials: store it *remotely* (in S3 or an equivalent, not on a laptop, so the team shares one authoritative copy); *lock* it during operations (so two people applying simultaneously don't corrupt it or race); *protect* it (it can contain sensitive values like passwords and connection strings, so it needs encryption and access control); and *back it up* (losing it is a serious recovery problem). The state file is the single most operationally sensitive artifact in a Terraform setup, which is why remote state with locking is a standard requirement.

4. Why should you always review the plan before applying, and never make manual changes?

**Review the plan** because it's your preview and last line of defence against destructive changes. Terraform's `plan` shows exactly what `apply` will do — which resources it will create, modify, or *destroy* — before anything happens. A subtle config change (a renamed resource, a changed immutable attribute) can cause Terraform to plan a *destroy-and-recreate* of a production database, and the plan output ("will destroy aws_db_instance.prod") is your chance to catch that before it executes. Applying blind means discovering the destruction after it happens. **Never make manual changes** because they cause drift — the real infrastructure diverges from what the code declares, which undermines every IaC guarantee: the code is no longer the source of truth, reproducing the environment won't match, and the next `apply` may revert your manual change (or conflict with it) in surprising ways. Once you allow manual console tweaks, you're back to not knowing how production is really configured. The discipline is: all changes go through the code, reviewed and planned; the console is read-only. IaC's benefits hold only if the code genuinely reflects reality, which requires that reality only change through the code.

5. What is GitOps and how does it relate to IaC?

GitOps extends IaC's principles to operations: git becomes the single source of truth for both infrastructure *and* application deployments, and an automated agent (Argo CD, Flux) continuously watches the git repository and reconciles the actual system state (typically a Kubernetes cluster) to match what's declared in git. You make any change — infrastructure, deployment, config — by committing to git, reviewed through a pull request; the agent then applies it automatically, and crucially, detects and reverts drift (if the live cluster diverges from git, the agent corrects it back). It relates to IaC as the operational, continuously-reconciled version: IaC gives you declarative, version-controlled infrastructure that you `apply`, while GitOps automates the apply into a constant reconciliation loop and makes git the authoritative desired state that the system perpetually converges toward. It's essentially the Kubernetes reconciliation model (declare desired state, controllers make reality match) applied to your whole delivery process — combining the declarative source of truth of IaC, the automation of CI/CD, and continuous drift correction. The benefits are a fully auditable delivery process (every change is a reviewed git commit), automatic drift correction, and easy rollback (revert the commit).