System Design Fundamentals

"Design Twitter."

It is 2018. You are in a conference room at Google's San Francisco office. The interviewer writes three words on the whiteboard and steps back.

"Design Twitter."

They are not asking you to write code. They are not asking for a data structure or an algorithm. They want to watch how you think about building a system used by 400 million people simultaneously — a system that must accept 6,000 tweets per second, store them durably, and deliver them to followers across the world in under 200 milliseconds.

This question separates engineers who can code from engineers who can architect. System design is the discipline of making high-level structural decisions before a single line of production code is written.

---

What System Design Is

System design is the process of defining the architecture, components, interfaces, and data flows of a system to satisfy specified requirements — particularly non-functional requirements like scale, availability, and latency.

It answers questions that no individual class or function can answer:

• How do we handle 10 million users when our current system handles 10,000?
• What happens when our primary database server crashes at 3am?
• How do we deliver a video file to a user in Tokyo from servers in Virginia?
• How do we let 100 engineers deploy independently without breaking each other's work?

---

The System Design Process

Experienced engineers follow a consistent process when tackling system design questions — in interviews and in real projects.

Step 1 — Clarify Requirements: Before drawing anything, ask questions. What exactly must the system do? How many users? What is the read-to-write ratio? Do we need strong consistency or eventual consistency? Real interviews fail because candidates assume instead of ask.

Step 2 — Estimate Scale: Back-of-envelope calculations anchor the design in reality. A system for 1,000 users is radically different from one for 1 billion.

Step 3 — High-Level Design: Identify the major components and how data flows between them. Boxes and arrows.

Step 4 — Detailed Design: Deep-dive into the 2–3 most critical or interesting components.

Step 5 — Identify Bottlenecks: Where will the system fail? How does it recover? How does it scale?

---

Back-of-Envelope Estimation

Estimation is a required skill. Nobody expects exact numbers — they want to see how you reason.

Useful constants to memorise:

Twitter estimation example:

• 300 million DAU (Daily Active Users)
• Each user reads 20 tweets per day
• Read QPS = 300M × 20 / 86,400 = ~70,000 reads/second
• Each user writes 1 tweet per 5 days = 0.2 tweets/day
• Write QPS = 300M × 0.2 / 86,400 = ~700 writes/second
• Each tweet: 280 characters × 2 bytes UTF-16 = 560 bytes ≈ 1 KB with metadata
• Daily write storage = 700 writes/sec × 86,400 sec × 1 KB = ~58 GB/day

---

Key Non-Functional Requirements

These are the dimensions that determine how the system is built, not what it does.

Availability

The percentage of time a system is operational.

Amazon estimated in 2012 that every 100ms of added latency costs 1% in sales. Google found that a 500ms delay in search results causes a 20% drop in traffic. Availability and latency are not academic — they are directly tied to revenue.

Scalability

The ability to handle increasing load. There are two approaches:

• Vertical scaling: bigger machine — more RAM, faster CPU. Limited by the largest available hardware.
• Horizontal scaling: more machines. Theoretically unlimited, but requires stateless services and data partitioning.

Consistency

In a distributed system, all replicas should agree on the current state of data. But achieving perfect consistency means slower responses and lower availability.

The CAP Theorem

Eric Brewer's CAP theorem (2000, formally proven 2002): in a distributed system experiencing a network partition, you must choose between:

Real example: Amazon's DynamoDB (AP) allows a shopping cart to accept items even when some replicas are unavailable. Two replicas might briefly disagree on cart contents — Amazon accepts this eventual consistency trade-off because a cart that silently refuses items loses sales.

Latency

• Average latency can be misleading — 10% of your requests being slow means millions of users have bad experiences.
• p99 latency (99th percentile): the response time below which 99% of requests fall. This is what SREs actually monitor.
• p99.9 latency: critical for financial systems, real-time services.

---

Core System Design Components

Every large-scale system assembles from a standard toolkit of components:

---

Designing Twitter: High-Level Architecture

Applying the process to Twitter's core features (post tweet, read home timeline):

Functional requirements clarified: Post tweets, follow users, read home timeline (tweets from followed users, most recent first).

Scale estimates (from above): ~700 writes/sec, ~70,000 reads/sec — this is a read-heavy system (100:1 read/write ratio).

Key design decisions:

1. Pre-compute timelines (fan-out on write): when a user tweets, push the tweet ID to all followers' timeline caches immediately. Reading a timeline is a single Redis lookup — O(1).
2. Exception for celebrities: users with 10M+ followers (Obama, Taylor Swift) use fan-out on read — pushing to 10M caches per tweet is too expensive.
3. CDN for media: images and videos served from edge locations, not origin servers.
4. Eventual consistency for timeline: it is acceptable if a follower sees a new tweet 1–2 seconds after it is posted.

---

Key Takeaways

• System design is about architectural decisions at scale — the questions no algorithm can answer alone.
• The five-step process (clarify → estimate → high-level design → detailed design → bottlenecks) provides a framework for tackling any design problem.
• Back-of-envelope estimation is not about precision — it is about reasoning in the right order of magnitude.
• The CAP theorem forces a choice between consistency and availability during network partitions. Real systems (DynamoDB, Cassandra) are AP; banking systems are CP.
• p99 latency, not average latency, is what matters for user experience and SRE monitoring.
• Every large system uses the same component toolkit: CDN, load balancer, cache, message queue, database replicas — assembled differently for each problem.
• Twitter's 100:1 read-to-write ratio drives its core architectural choice: pre-compute timelines on write, serve on read.