java - Apache Flink : aligning watermark among parallel tasks

Question

我们正在使用带水印的周期性事件时间窗口。我们的 Flink 应用程序中目前有 4 个并行任务。

在流式传输过程中，所有 4 个任务的水印值都必须接近触发窗口事件。

For example;

Task 1 watermark value = 8

Task 2 watermark value = 1

Task 3 watermark value = 8

Task 4 watermark value = 8

任务 2 正在等待日志更新其水印。然而，这种情况可能发生在任务 2 的更新之前，我们希望在它之前触发窗口事件。

是否有任何机制可以在不等待其他任务的情况下对齐所有并行任务的水印或触发窗口事件？

Answer 1

这已经在@Til Rohrmann 的评论中得到了回答，主要答案是:

If you want to look for 3 senders which are the same person, then you should keyBy the person and then simply use a count window to generate the alerts. Maybe you also wanna filter out duplicate senders for the same person.

后续问题:

But we need to consider also the event time. For example 3 senders in 10 minutes(event time not process time)

后续回答:

Then you can use an event time window with a custom trigger which fires when it has seen 3 elements.

我想批判性的结论是:
如果你想触发某个东西的计数，keyBy 那个字段。