system-design

The data you collect is a liability as much as an asset. Regulations turn “we store everything forever” from a convenience into a legal risk with real fines.

Prerequisites: Encryption, Data Modeling Time to read: ~20 minutes

Why this matters to system design

Privacy regulation isn’t a legal footnote — it imposes hard technical requirements that shape your architecture:

You must be able to delete all of a user’s data on request — which contradicts append-only logs, backups, and denormalized copies.
You must keep EU data in the EU — which forces data residency and regional architecture.
You must know where every piece of personal data lives — which requires data inventory and lineage.
You must encrypt and access-control PII — covered in Encryption.

🚨 These requirements often conflict with designs that are otherwise sensible (event sourcing, aggressive replication, “log everything”), and reconciling them is a genuine design skill. GDPR fines reach 4% of global annual revenue — for a large company, billions — so this is not optional.

What is PII

Personally Identifiable Information — data that identifies a person. And it’s broader than people expect:

Direct: name, email, phone, national ID, passport number, address.
Indirect / quasi-identifiers: 🚨 data that identifies someone in combination — a birth date + ZIP code + gender uniquely identifies ~87% of Americans. IP addresses, device IDs, and cookies are PII under GDPR.
Sensitive (special category): health, biometrics, race, religion, sexual orientation, political views — extra protections and usually explicit consent required.

🚨 The quasi-identifier point is the subtle one: anonymizing by removing names isn’t enough if the remaining fields still re-identify people. True anonymization is hard, and “we removed the names” is not compliance.

GDPR’s core rights (and their technical demands)

GDPR (EU) is the strictest and most influential; CCPA (California), and many national laws follow similar principles. The rights that impose engineering work:

Right to erasure (“right to be forgotten”)

🚨 The one that breaks architectures. A user can demand deletion of all their personal data.

Why it’s hard:

Backups contain the data. You can’t easily delete one row from an immutable backup.
Event sourcing and append-only logs are fundamentally at odds with deletion.
Denormalized copies (CQRS read models, search indexes, caches, analytics warehouses fed by CDC) all hold the data.
Third parties you shared it with must also delete it.

Techniques:

🚨 Crypto-shredding — encrypt each user’s PII with a per-user key; “delete” by destroying the key, rendering the data unreadable everywhere it exists, including backups and event logs. The standard answer for deletion in append-only systems, and it must be designed in from the start. → Event Sourcing
Deletion propagation — a delete triggers deletion across every derived store, propagated by events.
Backup expiry — accept that backups retain data until they roll off (documented in your policy), and re-apply deletions on restore.
A data map — you can’t delete what you don’t know you have.

🎙️ “Right to erasure is the hard requirement. For our event-sourced ledger and backups I’d use crypto-shredding — PII encrypted per-user, delete the key to make it unreadable. For live stores, a deletion event propagates to every derived copy: search index, cache, warehouse.”

Right of access & portability

Users can request a copy of all their data, in a machine-readable format. 🚨 Requires you to locate everything about one user across all your stores — which, in a system that scattered PII across a dozen services and derived stores, is genuinely hard. This is an argument for centralizing or at least cataloguing PII.

You need a legal basis to process personal data (consent, contract, legitimate interest). Consent must be explicit, granular, and revocable — no pre-ticked boxes, and users can withdraw it. Technically: track consent per purpose, and honour withdrawal (stop processing, and often delete).

Breach notification

You must report a breach to authorities within 72 hours. 🚨 This requires detection (you can’t report what you didn’t notice) and knowing what was exposed — which again requires knowing where PII lives and having security monitoring.

Data residency and sovereignty

🚨 Increasingly a hard architectural constraint. Many laws require personal data to stay within a country or region — EU data in the EU (GDPR), and a growing number of national data-localization laws (India, China, Russia, and others).

The design impact:

You can’t use a single global database. You need regional deployments with data pinned to regions.
Multi-tenancy by region — route each user to their regional data store.
Cross-region features (global search, analytics) get complicated — you may need to aggregate without moving raw PII, or keep separate regional analytics.
Backups and disaster recovery must respect residency too — an EU backup can’t be stored in the US.

🎙️ “Data residency forces regional architecture — I’d shard by region so EU users’ data lives in EU infrastructure, including backups. Cross-region analytics would aggregate anonymized data rather than moving raw PII across borders.”

This is why shared-everything multi-tenancy struggles with compliance and per-region/per-cell isolation is often necessary. → Multi-Tenancy

Privacy by design

🚨 The principle GDPR encodes: build privacy in from the start, not as an afterthought. Concrete practices:

1. Data minimization. 🚨 The most powerful and underused principle: don’t collect what you don’t need. The safest PII is the PII you never stored — it can’t be breached, doesn’t need deletion, and carries no compliance burden. Before adding a field, ask whether you genuinely need it. Resist the “collect everything, we might use it later” instinct — later it’s a liability.

2. Purpose limitation. Use data only for the purpose it was collected for. Data collected for account setup shouldn’t be repurposed for ad targeting without new consent.

3. Retention limits. Don’t keep data forever. Set retention policies and auto-delete — “we delete inactive accounts after 2 years.” This bounds your exposure and your deletion burden.

4. Pseudonymization and anonymization.

Pseudonymization — replace direct identifiers with tokens; the mapping is stored separately and protected. Reversible if needed. Still PII (re-identifiable), but lower risk.
Anonymization — irreversibly remove the ability to identify. 🚨 Genuinely hard because of quasi-identifiers; k-anonymity, l-diversity, and differential privacy are the techniques, and naive approaches (just dropping names) don’t achieve it. True anonymized data falls outside GDPR, which is why it’s valuable — but achieving it is non-trivial.

5. Separate PII from other data. Storing PII in a dedicated, encrypted, access-controlled store (rather than scattered through every table) makes deletion, access requests, and residency far easier. A common pattern: a PII vault holding personal fields, referenced by token from everywhere else.

The specific system-design conflicts

🚨 Where privacy requirements clash with otherwise-good designs — the interesting part for interviews:

Design pattern	Conflict	Reconciliation
Event sourcing	Immutable log vs right to erasure	Crypto-shredding; keep PII out of events
CDC to warehouse	Copies PII everywhere, weaker controls	Mask/filter PII in the pipeline; deletion propagation
Aggressive caching	Cached PII persists after deletion	Short TTLs; cache invalidation on delete
Global single database	Data residency	Regional sharding
Denormalization / CQRS	PII in many read models	Deletion propagation to all projections
“Log everything”	PII in logs; logs are a breach target	Never log PII; redact at the logging layer
Long backup retention	Deleted data survives in backups	Crypto-shredding; documented backup lifecycle

🎙️ A strong interview move is to raise one of these unprompted: “One thing to flag — feeding the warehouse via CDC copies PII into a system with weaker access controls, and complicates GDPR deletion. I’d mask PII in the pipeline and propagate deletions to the warehouse.”

⚖️ Trade-offs

Choice	Gain	Cost
Data minimization	Less breach risk, less compliance burden	Less data for analytics/ML
Crypto-shredding	Deletion in append-only systems and backups	Key management; per-user keys
Regional architecture	Residency compliance	Complexity; cross-region features harder
PII vault / centralization	Easy deletion, access, control	A critical dependency; a lookup
Pseudonymization	Lower risk, still usable	Still PII; mapping to protect
Retention limits	Bounded exposure	Lost historical data

In the real world

GDPR fines have reached the hundreds of millions and beyond — Meta, Amazon, and Google have all faced nine- and ten-figure penalties. The 4%-of-revenue ceiling makes compliance a board-level concern, not just an engineering one.
Crypto-shredding is the accepted industry technique for reconciling immutability (event sourcing, blockchain, backups) with the right to erasure — encrypt per-user, delete the key. It’s the answer regulators and architects converge on.
Data residency has reshaped cloud architecture — every major provider now offers region pinning, sovereign cloud regions, and data-residency controls, because localization laws made single-global- database designs non-compliant for a growing share of the world.

🚨 Interview traps

Ignoring privacy entirely in a design handling personal data.
“We’ll just delete the row” for right to erasure — ignores backups, event logs, derived stores.
Not knowing crypto-shredding for deletion in append-only systems.
Feeding PII into analytics/logs without masking.
A single global database for a system with EU users (residency).
Treating anonymization as trivial — quasi-identifiers re-identify.
Collecting more data than needed — the opposite of minimization.

🎙️ Soundbites

“The data we collect is a liability as much as an asset. The strongest control is minimization — the PII we never stored can’t be breached, doesn’t need deletion, and carries no compliance burden.”
“Right to erasure is the hard requirement because of backups and append-only logs. I’d use crypto-shredding — PII encrypted per-user, and deletion is destroying the key, which makes it unreadable everywhere including backups — plus deletion events propagated to every derived store.”
“Data residency forces regional architecture — EU users’ data, including backups, stays in EU infrastructure. Cross-region analytics would aggregate anonymized data rather than move raw PII.”
“I’d flag that feeding the warehouse via CDC copies PII into a system with weaker controls and complicates deletion — so I’d mask PII in the pipeline and propagate erasure downstream.”
“Anonymization is harder than dropping names — quasi-identifiers like birth date plus ZIP re-identify most people. For anything I claim is anonymized, I’d use k-anonymity or differential privacy, not naive field removal.”

🛠️ Try it

1. Map the PII in a system you know. List every place a user’s email or name is stored: primary database, cache, search index, analytics warehouse, logs, backups, third-party services (email provider, analytics, error tracker). The list is always longer than expected — and that list is your right-to-erasure problem.

2. Implement crypto-shredding. Encrypt a user’s PII fields with a per-user key stored separately. Store the encrypted data (imagine it’s also in an immutable event log). Then “delete” the user by destroying their key, and confirm the data is now unreadable everywhere — without touching the event log. This makes the technique concrete.

3. Find PII in your logs. Grep your application logs for email addresses, names, or IDs. Most systems log PII somewhere they shouldn’t — and logs are a breach target with weaker controls. Then add redaction at the logging layer.

4. Test a right-to-access request. Try to assemble everything your system knows about one user, across all stores. Time how long it takes and how many places you had to look. That’s the operational cost of scattered PII, and the argument for a PII vault.

Check yourself

1. Why does the right to erasure conflict with common architectures?

Because "delete all of a user's personal data" contradicts several patterns that assume data persists. **Backups** are immutable snapshots — you can't surgically delete one user's rows from a backup taken last month. **Event sourcing and append-only logs** are fundamentally about never deleting — the log is the source of truth. **Denormalized copies** mean the same PII exists in search indexes, caches, CQRS read models, and analytics warehouses fed by CDC — deleting from the primary database leaves copies everywhere. And **third parties** you shared the data with must also delete it. So erasure isn't a single `DELETE` — it's a distributed operation across every store, plus a strategy for immutable data. The standard reconciliation is crypto-shredding (encrypt per-user, delete the key) for immutable stores and backups, plus deletion events propagated to every derived copy.

2. What is crypto-shredding and why is it the standard answer for deletion in append-only systems?

Encrypt each user's personal data with a key unique to that user, stored separately in a mutable key store, and "delete" the user by destroying their key — the encrypted data remains physically present in every store (including immutable event logs and backups) but becomes permanently unreadable, which regulators generally accept as erasure. It's the standard answer for append-only systems because it sidesteps the impossibility of deleting from immutable data: you don't modify the log or the backup at all, you just render the personal fields undecryptable. The critical requirement is that it must be designed in from the start — you need to identify personal fields, encrypt them per-user at write time, and manage per-user keys — because retrofitting it means re-encrypting existing history, which defeats the immutability you were working with. It reconciles "never delete" with "must be able to delete."

3. Why isn't removing names enough to anonymize data?

Because of quasi-identifiers — fields that individually don't identify anyone but *in combination* do. The classic result is that birth date, ZIP code, and gender together uniquely identify about 87% of the US population, even with names removed. So a "de-identified" dataset with those fields can be re-identified by cross-referencing with public records or other datasets. Similarly, IP addresses, device IDs, and detailed behavioural traces can single out individuals. True anonymization — irreversibly removing the ability to identify anyone, which takes the data *outside* GDPR — requires techniques like k-anonymity (ensure each record is indistinguishable from at least k-1 others), l-diversity, or differential privacy (adding calibrated noise). Naive field removal produces pseudonymized data (still PII, still regulated), not anonymized data, and claiming otherwise is both a compliance failure and a re-identification risk.

4. How does data residency affect system architecture?

It prevents a single global data store and forces regional isolation. Laws like GDPR and various national localization requirements mandate that personal data stay within a specific jurisdiction — EU users' data in the EU, and so on — including backups and disaster-recovery copies. So you must shard or partition by region, routing each user to their regional data store, and deploy regional infrastructure rather than one central database. Cross-region features become genuinely harder: global search, analytics, or a unified view of all users can't simply query one database, and you often must aggregate anonymized or derived data rather than moving raw PII across borders. This is a major reason shared-everything multi-tenancy struggles with compliance and per-region cell architectures are used — residency is a hard constraint that shapes the whole topology, not a setting you toggle.

5. Why is data minimization described as the most powerful privacy control?

Because the safest data is the data you never collected. Every piece of PII you store is a liability across its entire lifecycle: it can be breached, it must be secured and access-controlled, it must be deletable on request, it must respect residency, it must be included in access-request exports, and it carries regulatory obligations. Data you didn't collect has none of these costs — there's nothing to breach, delete, protect, or account for. So not collecting a field is more effective than any control you could apply to it after collecting it. It's underused because the instinct is to "collect everything, we might need it later," but that "might" is speculative value weighed against concrete, ongoing liability and risk. Before adding any personal field, the question is whether you genuinely need it for a current purpose — and if not, not collecting it is the strongest privacy decision available.