Opening Scenario
Your analytics team currently writes 30 Spark batch jobs that run hourly on Parquet files in S3 and land aggregates in ClickHouse. The CFO wants the same numbers updated within seconds, not hours. The team lead asks: "Can we just rewrite these as streaming pipelines without retraining 12 analysts on Java?" You evaluate Flink SQL. It looks like the same SQL the analysts already write. But the moment you start a proof-of-concept, you hit oddly-named tables (FOR SYSTEM_TIME AS OF), warnings about "non-deterministic updates," queries that work in batch mode but fail in streaming, and confusion about whether SELECT * FROM topic is even legal. You need to understand what Flink SQL actually is, where it differs from batch SQL, and what production tradeoffs come with the "just write SQL" promise.
Question Progression
1. What does it mean to write SQL over a stream?
Question. SQL was designed for tables: finite, indexed, queryable in any order. Streams are unbounded sequences of events.…
In the full tutorial
- When to Reach for This