Wide-Column Store (Cassandra, HBase) — AP, Massive Write Throughput

Tier 1 — Building Block

1. What Is It?

A wide-column store organizes data as rows identified by a row key, where each row can have a different set of columns, and columns are grouped into column families. Unlike relational databases where all rows in a table have the same schema, wide-column stores allow each row to have a unique set of columns — and individual columns within a row are independently versioned, compressed, and stored.

The defining characteristics are: extreme write throughput (millions of writes/sec across a cluster), AP semantics (Available + Partition Tolerant from CAP), linear horizontal scalability, and no single point of failure. The trade-off: no JOINs, no secondary indexes (or limited ones), no multi-row ACID transactions, and a data model that must be designed around query patterns rather than data relationships.

Cassandra is the most widely used wide-column store. HBase is Hadoop-integrated (built on HDFS), more commonly found in data engineering pipelines.

2. How It Works

Cassandra Data Model

Cassandra's data model hierarchy: Keyspace → Table → Partition → Row → Column.

Keyspace: social_network
  Table: user_events
    Partition Key: user_id  ← determines which node(s) store this data
    Clustering Key: event_timestamp  ← determines order within the partition
    Columns: event_type, metadata

All rows with the same user_id are stored on the same node (the partition), sorted by event_timestamp. This enables efficient range queries within a partition (SELECT ... WHERE user_id=42 AND event_timestamp > '2024-01-01') but not across partitions (no efficient SELECT WHERE event_type='click').

-+ResetFullscreen

Write Path

-+ResetFullscreen

• CommitLog: A Write-Ahead Log (WAL) — every write is recorded here first, so it can be replayed on crash recovery
• MemTable: In-memory sorted write buffer. When it fills up, it's flushed to disk as an SSTable. This is the LSM tree pattern — converting random writes into sequential disk writes for higher throughput (see WAL & LSM Trees building block for details).
• SSTable: Sorted String Table — an immutable, sorted file on disk. Once written, never modified.
• W (write consistency level): How many replica ACKs the coordinator waits for before acknowledging to the client (configurable: ONE, QUORUM, ALL)

Read Path

-+ResetFullscreen

• R (read consistency level): How many replicas the coordinator reads from before returning (ONE = fast, may be stale; QUORUM = balanced; ALL = strong but slow)
• Read repair: When a coordinator reads from multiple replicas and detects inconsistency, it asynchronously sends corrections to stale replicas

Consistency Levels and CAP Trade-Off

Replication Factor (RF) = 3
N = 3 replicas per partition

Strong consistency: W + R > N
  QUORUM write (2 acks) + QUORUM read (2 acks) = 4 > 3 ✓ → Strong consistency

High availability: ONE write + ONE read → Fast, but may return stale data

3. Variants & Comparisons

Cassandra vs. DynamoDB

4. When to Use It (and When NOT To)

Use Cassandra When:

• Write throughput > 100K/sec: Cassandra scales writes linearly — add nodes, get proportional throughput
• Append-mostly workloads: Event logs, time-series, activity streams, IoT sensor data — new data added, rarely updated
• Global multi-datacenter: Cassandra's LOCAL_QUORUM consistency enables multi-datacenter deployments with strong local consistency
• Time-series data with time-range queries: Model as (device_id, timestamp) → values; partition by device, cluster by timestamp; range queries within a partition are efficient
• No JOINs needed: Your access patterns are all partition-key lookups, no relational queries

Decision triggers:

• "If writes > 100K/sec sustained → Cassandra or sharded SQL"
• "If data is time-series (metrics, events, logs) → Cassandra with (entity_id, timestamp) data model"
• "If you need zero-downtime global multi-datacenter writes → Cassandra with LOCAL_QUORUM"

Do NOT Use Cassandra When:

• Ad-hoc queries: SELECT WHERE arbitrary_column = X requires ALLOW FILTERING (full cluster scan) — catastrophically slow
• Relational data with JOINs: No JOIN support; must denormalize aggressively
• Multi-row ACID transactions: Cassandra's lightweight transactions (LWT using Paxos) exist but are expensive and limited. For real ACID transactions, use PostgreSQL.
• Low cardinality partition key: SELECT ... WHERE country='US' — US gets 40% of all rows on one partition. Cassandra partitions must be large enough to distribute load evenly but not one giant "hot" partition (max practical partition size ~100MB).

Anti-patterns:

• Giant partitions: A partition with 100M rows is slow to read, slow to compact, and causes GC pressure. Bucket your data: instead of (user_id, event_timestamp) producing huge partitions for power users, use (user_id, year_month, event_timestamp) to cap partition size per month.
• Too many secondary indexes: Cassandra secondary indexes are local (per node, not global). A query on a secondary index requires reading from ALL nodes — full cluster scatter-gather. Use materialized views or denormalized tables instead.
• Reads without partition key: Any query that doesn't filter by the full partition key becomes a full cluster scan. Design your tables around your query patterns first.
• Expecting SQL-like flexibility: Cassandra requires schema design upfront around specific access patterns. Changing your access pattern often requires creating a new table (denormalized copy of the data).

5. Real-World Usage

Netflix (Cassandra for viewing history): Netflix stores viewing history for ~260M subscribers in Cassandra. Every play, pause, resume, and completion event is a Cassandra write. The data model: partition by (profile_id), cluster by (video_id, event_timestamp). At peak, Netflix processes ~1 million writes/sec across their Cassandra clusters globally. They store trillions of rows. Netflix chose Cassandra for its multi-datacenter replication (their 3-region AWS architecture) and write throughput — they cannot afford to lose a viewing history write.

Discord (Cassandra for message storage, then ScyllaDB): Discord stored billions of messages in Cassandra with the schema (channel_id, message_id DESC) — partition by channel, cluster by message ID (Snowflake ID, descending for "newest first" queries). Discord served ~120 billion messages/year from Cassandra at peak. When read latency degraded due to compaction pressure and GC pauses on the JVM, Discord migrated to ScyllaDB (a C++ reimplementation of Cassandra) — keeping the same CQL API but achieving better and more predictable latency.

Apple (Cassandra at extreme scale): Apple runs one of the largest Cassandra deployments in the world — reportedly ~75,000+ Cassandra nodes, storing data for iCloud, iMessage, Siri, and other services. Apple's scale illustrates Cassandra's linear scalability: adding nodes proportionally increases both throughput and storage capacity.

6. Interview Cheat Sheet

Key sentences to demonstrate depth:

1. "Cassandra is the right choice when write throughput exceeds what a single SQL node can handle, the data model is append-mostly (events, time-series), and you need multi-datacenter replication. It trades JOINs, ad-hoc queries, and ACID for linear write scalability."
2. "The most important Cassandra design decision is the partition key — it determines data distribution. Bad partition key = hot partitions. The partition key must be high-cardinality, evenly distributes load, and must be in every WHERE clause you care about."
3. "Cassandra's consistency is tunable: W + R > N guarantees strong consistency. QUORUM + QUORUM on RF=3 (2+2>3) is the most common production choice — strong consistency with tolerance for one node failure."
4. "Cassandra has no joins — you must denormalize. One query pattern = one table. If you need to look up the same data by both user_id and email, you have two tables: one partitioned by user_id, one by email."
5. "Giant partitions are a Cassandra anti-pattern — partitions larger than ~100MB cause slow compaction and GC pauses. Bucket time-series data by time period (e.g., (user_id, year_month, timestamp)) to keep partition sizes bounded."

Common follow-up questions:

Connections to other building blocks:

• LSM Trees / WAL: Cassandra's write path is a classic LSM implementation — CommitLog (WAL) + MemTable + SSTable compaction. All write performance properties of LSM apply.
• Consistent Hashing: Cassandra distributes partitions using consistent hashing on the partition key. The token ring is a direct application of consistent hashing with VNodes.
• CAP Theorem: Cassandra is the canonical AP database example. With tunable consistency, it can behave as CP (ALL consistency) but at the cost of availability.
• Sharding: Cassandra's token ring is automatic horizontal sharding. Each node owns a token range; data is sharded across all nodes without any application-layer routing.
• Message Queues: Cassandra is sometimes used as a queue backing store (but with limitations — Cassandra's tombstone problem makes it poorly suited for high-delete-rate queue patterns). Kafka is better for queuing.