Opening Scenario
You operate a real-time fraud-scoring pipeline. A Kafka topic transactions streams every card swipe at ~50K events/sec. To score each transaction, the pipeline needs the cardholder's current risk tier, country of residence, and account-creation date — fields stored in a customers table in your operational Postgres. You start by having the Flink job synchronously query Postgres for each event. P99 latency balloons to 4 seconds, Postgres CPU pins at 95%, and your DBAs send you a polite Slack message. You need a fundamentally different pattern: the streaming side is fast, but the dimensional side is essentially a table. How do you join them without taking down the database?
Question Progression
1. Why is joining a stream to a table different from joining two streams?
Question. In stream-stream-joins the cost was buffering both sides in state. Here one side is a Postgres table that already lives somewhere. What's actually different about this join?
In the full tutorial
- When to Reach for This