Approach & Numbers
- Requirements (functional + non-functional).
- Capacity math (QPS, storage, bandwidth).
- High-level diagram.
- Deep dive the hard parts + trade-offs.
Cheat numbers
1 day ≈ 10⁵ s. 1M/day ≈ 12/s, 1B/day ≈ 12K/s. Read:write often 100:1.
URL Shortener
- Encode unique ID in base62; len 7 = 62⁷ ≈ 3.5T.
- Key gen: distributed counter (no collisions) > hash+truncate.
- Store: KV (DynamoDB/Cassandra), code → longURL.
- Redirect: cache-first (Redis); 301 caches, 302 for analytics.
News Feed
| Push (write) | Pull (read) |
|---|---|
| Cheap reads | Expensive reads |
| Costly for celebs | Cheap writes |
- Hybrid: push normal users, pull celebrities at read.
- Store post IDs in feed (Redis lists), hydrate content on read.
- Rank by recency + affinity + predicted engagement.
Rate Limiter
| Algo | Note |
|---|---|
| Token bucket | Allows bursts |
| Leaky bucket | Smooths output |
| Fixed window | Edge burst 2× |
| Sliding window counter | Accurate + cheap |
- Redis shared counters; atomic Lua for check+incr.
INCR+EXPIRE; return429+Retry-After.- Extreme scale: local buckets + reconcile. Fail open on Redis down.
Chat System
- WebSocket persistent conn per client.
- Session registry (Redis): user → WS server, for routing.
- Persist first, then push; client ACKs, server retries unacked.
- Ordering: per-conversation sequence / time-sortable ID.
- Store: partition by conversation ID (Cassandra/HBase).
- Presence: heartbeat TTL. Offline → APNs/FCM push.
Cross-Cutting
- Cache the read path.
- Shard by natural key (code / conversation / user).
- Precompute when reads dominate.
- Prefer eventual consistency where tolerable.
- Always ask: what breaks at 100× and how does it degrade?
Practice
- Twitter, web crawler, YouTube
- Uber, distributed cache, Google Drive