Package org.apache.iceberg.flink

Class FlinkWriteConf

java.lang.Object

org.apache.iceberg.flink.FlinkWriteConf

public class FlinkWriteConf
extends java.lang.Object

A class for common Iceberg configs for Flink writes.

If a config is set at multiple levels, the following order of precedence is used (top to bottom):

1. Write options
2. flink ReadableConfig
3. Table metadata

The most specific value is set in write options and takes precedence over all other configs. If no write option is provided, this class checks the flink configuration for any overrides. If no applicable value is found in the write options, this class uses the table metadata.

Note this class is NOT meant to be serialized.

Constructor Summary

Constructors

Constructor	Description
FlinkWriteConf(Table table, java.util.Map<java.lang.String,java.lang.String> writeOptions, org.apache.flink.configuration.ReadableConfig readableConfig)

Method Summary

Modifier and Type	Method	Description
java.lang.String	avroCompressionCodec()
java.lang.String	avroCompressionLevel()
java.lang.String	branch()
FileFormat	dataFileFormat()
DistributionMode	distributionMode()
java.lang.String	orcCompressionCodec()
java.lang.String	orcCompressionStrategy()
boolean	overwriteMode()
java.lang.String	parquetCompressionCodec()
java.lang.String	parquetCompressionLevel()
double	rangeDistributionSortKeyBaseWeight()
StatisticsType	rangeDistributionStatisticsType()
java.time.Duration	tableRefreshInterval()	NOTE: This may be removed or changed in a future release.
long	targetDataFileSize()
boolean	upsertMode()
int	workerPoolSize()
java.lang.Integer	writeParallelism()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FlinkWriteConf

public FlinkWriteConf(Table table,
                      java.util.Map<java.lang.String,java.lang.String> writeOptions,
                      org.apache.flink.configuration.ReadableConfig readableConfig)

Method Detail

overwriteMode

public boolean overwriteMode()

upsertMode

public boolean upsertMode()

dataFileFormat

public FileFormat dataFileFormat()

targetDataFileSize

public long targetDataFileSize()

parquetCompressionCodec

public java.lang.String parquetCompressionCodec()

parquetCompressionLevel

public java.lang.String parquetCompressionLevel()

avroCompressionCodec

public java.lang.String avroCompressionCodec()

avroCompressionLevel

public java.lang.String avroCompressionLevel()

orcCompressionCodec

public java.lang.String orcCompressionCodec()

orcCompressionStrategy

public java.lang.String orcCompressionStrategy()

distributionMode

public DistributionMode distributionMode()

rangeDistributionStatisticsType

public StatisticsType rangeDistributionStatisticsType()

rangeDistributionSortKeyBaseWeight

public double rangeDistributionSortKeyBaseWeight()

workerPoolSize

public int workerPoolSize()

branch

public java.lang.String branch()

writeParallelism

public java.lang.Integer writeParallelism()

tableRefreshInterval

@Experimental
public java.time.Duration tableRefreshInterval()

NOTE: This may be removed or changed in a future release. This value specifies the interval for refreshing the table instances in sink writer subtasks. If not specified then the default behavior is to not refresh the table.

Returns:

the interval for refreshing the table in sink writer subtasks