Distributed Cache (Memcached)

Phase 1 — Clarify & Scope

Interviewer: Design a distributed cache — think Memcached at Facebook scale. What do you want to clarify?

Candidate: A few clarifying questions:

1. What's the primary use case — database query caching, session storage, or computed result caching?
2. What's the read/write ratio?
3. What are the data types — simple key-value, or richer structures (lists, sets, sorted sets)?
4. Do we need persistence, or is the cache purely ephemeral?
5. What's the scale — requests per second, total data size?

Interviewer: Database query caching for a social network. Primarily reads (100:1 read-to-write ratio). Simple key-value only — values are serialized JSON objects. No persistence needed. Scale: 10 million requests/sec, 10 TB of cached data, 100 μs p99 latency SLA.

Candidate: Let me size this:

$\text{Read rate} = \frac{10\text{M req/sec} \times 100}{101} \approx 9.9\text{M reads/sec}$

$\text{Write rate} \approx 99{,}000 \text{ writes/sec}$

With 10 TB data and 64 GB RAM per cache node:

$\text{Cache nodes} = \frac{10 \text{ TB}}{64 \text{ GB}} \approx 156 \text{ nodes}$

Per-node load: $\frac{10\text{M}}{156} \approx 64{,}102 \text{ req/sec per node}$

Memcached handles 200K+ req/sec on modern hardware — we have ample headroom per node.

Interviewer: Good. What are the core requirements?

Candidate:

• Store and retrieve values by key (GET, SET, DELETE)
• Sub-millisecond reads under normal load, p99 < 100 μs
• Horizontal scaling — add nodes without downtime
• High availability — tolerate single node failure without serving stale data