Package org.apache.iceberg.flink.source.reader

Class ColumnStatsWatermarkExtractor

java.lang.Object

org.apache.iceberg.flink.source.reader.ColumnStatsWatermarkExtractor

All Implemented Interfaces:

Serializable, SplitWatermarkExtractor

@Internal
public class ColumnStatsWatermarkExtractor
extends Object
implements SplitWatermarkExtractor, Serializable

SplitWatermarkExtractor implementation which uses an Iceberg timestamp column statistics to get the watermarks for the IcebergSourceSplit. This watermark is emitted by the WatermarkExtractorRecordEmitter along with the actual records.

See Also:

• Serialized Form

Constructor Summary

Constructors

Constructor	Description
ColumnStatsWatermarkExtractor(Schema schema, String eventTimeFieldName, TimeUnit timeUnit)	Creates the extractor.

Method Summary

Modifier and Type	Method	Description
long	extractWatermark(IcebergSourceSplit split)	Get the watermark for a split using column statistics.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

ColumnStatsWatermarkExtractor

public ColumnStatsWatermarkExtractor(Schema schema,
 String eventTimeFieldName,
 TimeUnit timeUnit)

Creates the extractor.

Parameters:

schema - The schema of the Table

eventTimeFieldName - The column which should be used as an event time

timeUnit - Used for converting the long value to epoch milliseconds

Method Details

extractWatermark

public long extractWatermark(IcebergSourceSplit split)

Get the watermark for a split using column statistics.

Specified by:

extractWatermark in interface SplitWatermarkExtractor

Parameters:

split - The split

Returns:

The watermark

Throws:

IllegalArgumentException - if there is no statistics for the column