Package org.apache.iceberg.flink.maintenance.operator

Class TriggerManager

java.lang.Object

org.apache.flink.api.common.functions.AbstractRichFunction

org.apache.flink.streaming.api.functions.KeyedProcessFunction<Boolean,TableChange,Trigger>

org.apache.iceberg.flink.maintenance.operator.TriggerManager

All Implemented Interfaces:

Serializable, org.apache.flink.api.common.functions.Function, org.apache.flink.api.common.functions.RichFunction, org.apache.flink.streaming.api.checkpoint.CheckpointedFunction

@Internal
public class TriggerManager
extends org.apache.flink.streaming.api.functions.KeyedProcessFunction<Boolean,TableChange,Trigger>
implements org.apache.flink.streaming.api.checkpoint.CheckpointedFunction

TriggerManager starts the Maintenance Tasks by emitting Trigger messages which are calculated based on the incoming TableChange messages. The TriggerManager keeps track of the changes since the last run of the Maintenance Tasks and triggers a new run based on the result of the TriggerEvaluator.

The TriggerManager prevents overlapping Maintenance Task runs using TriggerLockFactory.Lock. The current implementation only handles conflicts within a single job. Users should avoid scheduling maintenance for the same table in different Flink jobs.

The TriggerManager should run as a global operator. KeyedProcessFunction is used, so the timer functions are available, but the key is not used.

See Also:

• Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.streaming.api.functions.KeyedProcessFunction

org.apache.flink.streaming.api.functions.KeyedProcessFunction.Context, org.apache.flink.streaming.api.functions.KeyedProcessFunction.OnTimerContext

Constructor Summary

Constructors

Constructor	Description
TriggerManager(TableLoader tableLoader, TriggerLockFactory lockFactory, List<String> maintenanceTaskNames, List<TriggerEvaluator> evaluators, long minFireDelayMs, long lockCheckDelayMs)

Method Summary

Modifier and Type	Method	Description
void	close()
void	initializeState(org.apache.flink.runtime.state.FunctionInitializationContext context)
void	onTimer(long timestamp, org.apache.flink.streaming.api.functions.KeyedProcessFunction<Boolean,TableChange,Trigger>.org.apache.flink.streaming.api.functions.KeyedProcessFunction.OnTimerContext ctx, org.apache.flink.util.Collector<Trigger> out)
void	open(org.apache.flink.configuration.Configuration parameters)
void	processElement(TableChange change, org.apache.flink.streaming.api.functions.KeyedProcessFunction<Boolean,TableChange,Trigger>.org.apache.flink.streaming.api.functions.KeyedProcessFunction.Context ctx, org.apache.flink.util.Collector<Trigger> out)
void	snapshotState(org.apache.flink.runtime.state.FunctionSnapshotContext context)

Methods inherited from class org.apache.flink.api.common.functions.AbstractRichFunction

getIterationRuntimeContext, getRuntimeContext, setRuntimeContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.api.common.functions.RichFunction

open

Constructor Details

TriggerManager

public TriggerManager(TableLoader tableLoader,
 TriggerLockFactory lockFactory,
 List<String> maintenanceTaskNames,
 List<TriggerEvaluator> evaluators,
 long minFireDelayMs,
 long lockCheckDelayMs)

Method Details

open

public void open(org.apache.flink.configuration.Configuration parameters)
          throws Exception

Specified by:

open in interface org.apache.flink.api.common.functions.RichFunction

Overrides:

open in class org.apache.flink.api.common.functions.AbstractRichFunction

Throws:

Exception

snapshotState

public void snapshotState(org.apache.flink.runtime.state.FunctionSnapshotContext context)
                   throws Exception

Specified by:

snapshotState in interface org.apache.flink.streaming.api.checkpoint.CheckpointedFunction

Throws:

Exception

initializeState

public void initializeState(org.apache.flink.runtime.state.FunctionInitializationContext context)
                     throws Exception

Specified by:

initializeState in interface org.apache.flink.streaming.api.checkpoint.CheckpointedFunction

Throws:

Exception

processElement

public void processElement(TableChange change,
 org.apache.flink.streaming.api.functions.KeyedProcessFunction<Boolean,TableChange,Trigger>.org.apache.flink.streaming.api.functions.KeyedProcessFunction.Context ctx,
 org.apache.flink.util.Collector<Trigger> out)
                    throws Exception

Specified by:

processElement in class org.apache.flink.streaming.api.functions.KeyedProcessFunction<Boolean,TableChange,Trigger>

Throws:

Exception

onTimer

public void onTimer(long timestamp,
 org.apache.flink.streaming.api.functions.KeyedProcessFunction<Boolean,TableChange,Trigger>.org.apache.flink.streaming.api.functions.KeyedProcessFunction.OnTimerContext ctx,
 org.apache.flink.util.Collector<Trigger> out)
             throws Exception

Overrides:

onTimer in class org.apache.flink.streaming.api.functions.KeyedProcessFunction<Boolean,TableChange,Trigger>

Throws:

Exception

close

public void close()
           throws IOException

Specified by:

close in interface org.apache.flink.api.common.functions.RichFunction

Overrides:

close in class org.apache.flink.api.common.functions.AbstractRichFunction

Throws:

IOException