Opening scenario
Your fraud-detection Flink job uses event-time tumbling windows of 1 minute. In dev with synthetic data, results emit promptly: each window closes a second after wall-clock crosses its end. In production with real traffic, the same job has windows that never emit, or windows that emit hours late, or — most confusingly — a sudden flood of all-at-once window results after long quiet periods. Operations is paged because the downstream alerting system thinks the pipeline is dead.
You check the Flink dashboard. The job is healthy. Records are flowing. Backlog is low. But the watermark for one source is sitting at 12:04:30 while wall clock is 13:47:00. Watermarks are the engine's clock; without them, event-time windows have no signal that any window is ever complete. To debug this, you need to understand what watermarks claim, where they come from, and what makes them stall.
Why does event-time windowing need an explicit clock signal?
You can…
In the full tutorial
- How is a watermark actually generated from incoming records?
- What does per-partition watermark behavior mean for the operator's effective watermark?
- How does the watermark interact with state cleanup?
- How do periodic vs punctuated watermarks differ, and when does the choice matter?
- What does a watermark dashboard look like in practice?
- Closing: When to reach for this