system-design

Design a CI/CD Platform (GitHub Actions / Jenkins / GitLab CI)

Difficulty: Tier 2 Asked at: GitHub, GitLab, Amazon, infra/platform teams Time budget: 45 min

A CI/CD platform runs your tests and deploys your code on every push. Under the hood it’s a distributed job-execution system: take a pipeline definition, schedule its stages onto a fleet of workers, run untrusted code in isolated environments, stream logs, and enforce dependencies between steps. It combines the job scheduler, queues, and a strong isolation/security angle (you’re running arbitrary user code).

Prerequisites: Design a Job Scheduler, Background Jobs, Deployment & Infra

1. Requirements

Functional:

On a trigger (git push, PR, schedule), run a pipeline defined in config (YAML): stages of jobs with dependencies (build → test → deploy).
Run jobs on a fleet of runners/workers; support parallelism and matrix builds.
Stream logs in real time; report status back (checks on the PR).
Cache dependencies/artifacts between runs; pass artifacts between stages.
Secrets management for deploy credentials.

Non-functional:

Isolation & security — jobs run untrusted arbitrary code; must not affect the host or other jobs. 🚨 The defining concern.
Scalable — many concurrent pipelines across many repos/orgs.
Reliable — a worker crash retries the job; no lost pipelines.
Reasonable latency — developers wait on CI; queue time matters.

Out of scope: the git hosting itself, the deployment targets’ internals.

2. Estimation

Thousands of orgs × many pushes/day → thousands of pipeline runs/sec at peak, each fanning into many jobs → a large, bursty job-execution load. Needs a scalable worker fleet + queue.
Jobs are heterogeneous (30s lint to 60-min integration test) and resource-hungry → schedule onto appropriately-sized runners.

3. High-level design

flowchart TB
    Trigger[Git push / PR / schedule] --> Orch[Pipeline Orchestrator<br/>parse config, build DAG]
    Orch --> Q[[Job queue]]
    Q --> Runner1[Runner / Worker<br/>isolated sandbox]
    Q --> Runner2[Runner / Worker]
    Runner1 -->|logs stream| LogSvc[Log streaming]
    Runner1 -->|status| Orch
    Orch --> Status[Status / checks back to git]
    Cache[(Artifact + dependency cache)] --> Runner1
    Secrets[(Secret store)] --> Runner1

Orchestrator parses the pipeline config into a DAG of jobs (respecting dependencies), enqueues ready jobs, tracks completion, and enqueues the next stage. Runners pull jobs, execute them in isolated sandboxes, stream logs, and report status. Artifacts/caches and secrets are provided to jobs.

4. Deep dives

4a. The pipeline as a DAG

🚨 A pipeline is a directed acyclic graph of jobs. Stages have dependencies (test needs build; deploy needs test). The orchestrator schedules a job only when its dependencies succeed, runs independent jobs in parallel, and fails fast (a failed dependency skips dependents). This DAG scheduling — with fan-out (parallel jobs) and fan-in (a stage waiting on several) — is the core control logic. It’s the job scheduler with dependencies.

4b. Isolation — running untrusted code safely

🚨 The security heart. User pipelines run arbitrary code; it must not compromise the host, other tenants’ jobs, or secrets. Isolation options (increasing strength):

Containers — the common choice; each job in a fresh container, resource-limited (cgroups), namespaced.
VMs / microVMs (Firecracker) — stronger isolation for untrusted multi-tenant code; near-container speed.
Ephemeral runners — a fresh, throwaway environment per job so no state leaks between runs and a compromised job is destroyed after. 🚨 “Ephemeral + sandboxed per job” is the key security stance. Combine with least-privilege (jobs get only the secrets/permissions they need). (Deployment & Infra)

4c. Scheduling onto the runner fleet

Match jobs to runners by required resources/labels (a GPU job → GPU runner; Linux vs Windows). Autoscale the fleet on queue depth (spin up runners for a burst, tear down when idle — CI load is spiky). Fair scheduling across tenants so one org’s huge pipeline doesn’t starve others. Queue time is a key UX metric.

4d. Log streaming

Developers watch logs live. Runners stream job output (chunked) to a log service that fans out to watching browsers (WebSocket/SSE) and stores logs durably for later viewing. Handle large, fast log output without overwhelming storage or the UI (buffer, truncate absurd output).

4e. Caching & artifacts

Re-downloading dependencies every run is slow; cache them (keyed by lockfile hash) in object storage and restore per run. Artifacts (build outputs) produced by one stage are stored and passed to dependent stages. This caching is often the biggest CI speed win. (Object Storage)

4f. Reliability & secrets

Worker crash: the job is retried (lease/visibility timeout like the job scheduler); idempotent where possible (though builds have side effects — usually just re-run).
Secrets: stored encrypted, injected into jobs at runtime, never logged, scoped to the job. Prevent exfiltration by untrusted code (a real threat — e.g. masking secrets in logs, restricting on forks).

5. Bottlenecks & scaling further

Job execution load → queue + autoscaling runner fleet.
Isolation/security → containers/microVMs, ephemeral per-job environments, least privilege.
DAG scheduling → orchestrator runs independent jobs in parallel, respects dependencies.
Slow builds → dependency/artifact caching (often the biggest win).
Fairness → per-tenant scheduling/quotas so one org can’t starve others.
Log volume → streamed, buffered, durably stored with limits.

6. Trade-off summary

Decision	Chosen	Alternative	Why
Isolation	Ephemeral containers/microVMs per job	Shared runners	Untrusted code must not leak/compromise
Pipeline model	DAG with parallel + dependencies	Linear script	Parallelism + correct ordering + fail-fast
Fleet	Autoscaling on queue depth	Fixed pool	CI load is spiky; balance cost vs queue time
Speed	Dependency/artifact caching	Fresh every time	Biggest build-time win
Scheduling	Fair per-tenant	FIFO	One org shouldn’t starve others

7. Follow-up questions

How do you safely run arbitrary untrusted user code?

By isolating every job in an ephemeral, sandboxed environment with least privilege, so a malicious or buggy pipeline can't affect the host, other tenants' jobs, or secrets. Each job runs in a fresh container (resource- limited via cgroups and namespaced) or, for stronger multi-tenant isolation, a lightweight microVM like Firecracker that gives VM-grade separation at near-container speed. The environment is ephemeral — created for that one job and destroyed afterward — so no state, credentials, or compromise leaks between runs, and a job that's exploited is thrown away when it finishes. On top of isolation you apply least privilege: jobs receive only the secrets and permissions they actually need, secrets are injected at runtime and masked from logs to prevent exfiltration, and untrusted contexts (like pull requests from forks) get further restricted access. The combination — per-job ephemeral sandboxes, strong isolation boundaries, and scoped least-privilege credentials — is what lets a shared platform run everyone's arbitrary code without one tenant endangering another or the platform itself. Isolation is the defining concern of a CI/CD system precisely because running untrusted code is its core function.

Why model a pipeline as a DAG rather than a linear list of steps?

Because real pipelines have both parallelism and dependencies that a linear list can't express efficiently or correctly. Some jobs depend on others (deploy needs tests to pass, tests need the build), while others are independent and should run simultaneously (lint, unit tests, and a security scan can all run at once). A directed acyclic graph captures exactly this: edges encode "must run after," so the orchestrator runs a job only when its dependencies have succeeded, executes independent jobs in parallel to minimize total time, waits at fan-in points where a stage needs several predecessors, and fails fast by skipping dependents when a dependency fails. A linear script would force everything to run sequentially even when it needn't (slow) or would lack the structure to know what depends on what (incorrect ordering, no fail-fast). The DAG is the natural model for "run these jobs respecting their dependencies while maximizing parallelism," which is the core scheduling logic of CI/CD — essentially the job-scheduler problem with dependency edges.

CI load is spiky. How do you size the runner fleet?

By autoscaling the runner fleet based on queue depth rather than provisioning a fixed pool. CI demand is bursty — quiet overnight, then a flood when the workday starts or a big merge triggers many pipelines — so a fixed fleet is either wastefully idle most of the time or too small during peaks, leaving developers waiting in a long queue. Instead you watch the job queue and scale runners up when it grows (spinning up more workers, often on cheap/spot capacity) and tear them down when it drains, matching capacity to real-time demand. You schedule jobs onto runners by matching required resources and labels (a GPU or Windows job goes to a capable runner), and you enforce fair per-tenant scheduling and quotas so one organization's giant pipeline can't monopolize the fleet and starve everyone else. Queue time is a key user-experience metric, so the goal is to keep it low during bursts without paying for idle capacity in the troughs — which elastic, queue-driven autoscaling with fair scheduling delivers.

What's usually the biggest lever for making builds faster?

Caching dependencies and reusing artifacts between runs. A large fraction of build time is often spent re-downloading and re-installing the same dependencies (packages, modules, base images) on every run, and recompiling unchanged code, so caching those — keyed by a hash of the lockfile or inputs so the cache is valid — and restoring them at the start of a job eliminates most of that repeated work. Similarly, artifacts produced by one stage (compiled binaries, built images) are stored and passed to dependent stages rather than rebuilt, and incremental builds reuse prior outputs where inputs haven't changed. Combined with running independent jobs in parallel via the DAG, caching typically gives the biggest wall-clock reduction because it attacks the repeated, redundant work that dominates naive pipelines. The cache and artifacts live in fast object storage and are scoped per repo/branch to stay correct. So while parallelism and bigger runners help, dependency/artifact caching is usually the single highest-impact optimization.

8. What junior / mid / senior answers look like

Junior: runs build steps sequentially on a server. Works for one project; no isolation (security hole), no parallelism, no scaling.
Mid: orchestrator builds a job DAG, a queue feeds a runner fleet, jobs run in containers, logs stream, status reports back, dependencies cached.
Senior: emphasizes ephemeral per-job isolation (containers/microVMs) and least-privilege secrets as the security core, DAG scheduling with parallelism and fail-fast, autoscaling the fleet on queue depth with fair per-tenant scheduling, caching as the top speed lever, and lease-based retry for worker crashes — framing it as a secure, multi-tenant distributed job-execution system.