S3-like Object Storage

Phase 1 — Clarify & Scope

Interviewer: Design an S3-like object storage system. What do you want to clarify first?

Candidate: A few questions to scope this:

1. What's the scale — how many objects, and what size distribution? A few large video files or billions of small files?
2. Access patterns: mostly write-once-read-many, or frequent overwrites?
3. Durability and availability targets?
4. Do we need multi-region replication or single-region?
5. Features to support: versioning, lifecycle policies, presigned URLs, multipart upload?

Interviewer: 100 billion objects total. Size distribution: 70% under 1 MB (user uploads, documents), 20% between 1–100 MB, 10% over 100 MB (videos, backups). Mostly write-once-read-many. Target: 11 nines of durability (99.999999999%), 99.99% availability. Single region with 3 AZs. Support multipart upload and presigned URLs. Versioning is a stretch goal.

Candidate: Let me size this:

$\text{Average object size} \approx 0.7 \times 0.5\text{MB} + 0.2 \times 10\text{MB} + 0.1 \times 200\text{MB} = 22.35 \text{ MB average}$

$\text{Total storage} = 100 \times 10^9 \times 22.35 \text{ MB} \approx 2.2 \text{ EB}$

For 11 nines durability with 3-AZ erasure coding at 6+3 (Reed-Solomon):

$\text{Storage overhead} = \frac{6+3}{6} = 1.5\times \Rightarrow 2.2 \text{ EB} \times 1.5 = 3.3 \text{ EB physical}$

Compared to 3× replication: 3 × 2.2 EB = 6.6 EB — erasure coding saves 50%.

Interviewer: Good. Request volume?

Candidate: