Throughput vs Latency Tuning

Opening Scenario

You run a Kafka pipeline that ingests user-interaction events from a busy web tier. The product team has two complaints in the same standup. The growth analyst says yesterday's nightly aggregation job missed its SLA because the upstream topic was lagging by 40 minutes — they want more throughput. The notifications team says push notifications triggered by these events are arriving 8 seconds after the click — they want lower latency. You look at the producer and consumer configs and realize most knobs do one at the expense of the other. Your job is to tune for both teams without lying to either of them.

---

Question Progression

1. Why do throughput and latency tend to trade off in the first place?

Question. A teammate says "just make it faster." What are they actually asking for, and why can't you give them both at once for free?

Reasoning. Throughput is records…