Package org.apache.iceberg.flink.maintenance.operator

Class ListFileSystemFiles

java.lang.Object
org.apache.flink.api.common.functions.AbstractRichFunction
org.apache.flink.streaming.api.functions.ProcessFunction<Trigger,String>
org.apache.iceberg.flink.maintenance.operator.ListFileSystemFiles
All Implemented Interfaces:
Serializable, org.apache.flink.api.common.functions.Function, org.apache.flink.api.common.functions.RichFunction
@Internal public class ListFileSystemFiles extends org.apache.flink.streaming.api.functions.ProcessFunction<Trigger,String>
Recursively lists the files in the `location` directory. Hidden files, and files younger than the `minAgeMs` are omitted in the result.
See Also:

  • Nested Class Summary

    Nested classes/interfaces inherited from class org.apache.flink.streaming.api.functions.ProcessFunction

    org.apache.flink.streaming.api.functions.ProcessFunction.Context, org.apache.flink.streaming.api.functions.ProcessFunction.OnTimerContext

  • Constructor Summary

    Constructors
    Constructor
    Description
    ListFileSystemFiles(String taskName, int taskIndex, TableLoader tableLoader, String location, long minAgeMs, boolean usePrefixListing)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    close()
     
    void
    open(org.apache.flink.api.common.functions.OpenContext openContext)
     
    void
    processElement(Trigger trigger, org.apache.flink.streaming.api.functions.ProcessFunction<Trigger,String>.org.apache.flink.streaming.api.functions.ProcessFunction.Context ctx, org.apache.flink.util.Collector<String> out)
     

    Methods inherited from class org.apache.flink.streaming.api.functions.ProcessFunction

    onTimer

    Methods inherited from class org.apache.flink.api.common.functions.AbstractRichFunction

    getIterationRuntimeContext, getRuntimeContext, setRuntimeContext

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Constructor Details

    • ListFileSystemFiles

      public ListFileSystemFiles(String taskName, int taskIndex, TableLoader tableLoader, String location, long minAgeMs, boolean usePrefixListing)

  • Method Details

    • open

      public void open(org.apache.flink.api.common.functions.OpenContext openContext) throws Exception
      Specified by:
      open in interface org.apache.flink.api.common.functions.RichFunction
      Overrides:
      open in class org.apache.flink.api.common.functions.AbstractRichFunction
      Throws:
      Exception

    • processElement

      public void processElement(Trigger trigger, org.apache.flink.streaming.api.functions.ProcessFunction<Trigger,String>.org.apache.flink.streaming.api.functions.ProcessFunction.Context ctx, org.apache.flink.util.Collector<String> out) throws Exception
      Specified by:
      processElement in class org.apache.flink.streaming.api.functions.ProcessFunction<Trigger,String>
      Throws:
      Exception

    • close

      public void close() throws Exception
      Specified by:
      close in interface org.apache.flink.api.common.functions.RichFunction
      Overrides:
      close in class org.apache.flink.api.common.functions.AbstractRichFunction
      Throws:
      Exception