Package org.apache.iceberg.spark

Class FileRewriteCoordinator

  • public class FileRewriteCoordinator
extends java.lang.Object

    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void clearRewrite​(Table table, java.lang.String fileSetId)  
      java.util.Set<F> fetchNewFiles​(Table table, java.lang.String fileSetId)  
      java.util.Set<java.lang.String> fetchSetIds​(Table table)  
      static FileRewriteCoordinator get()  
      void stageRewrite​(Table table, java.lang.String fileSetId, java.util.Set<F> newFiles)
      Called to persist the output of a rewrite action for a specific group.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Method Detail

      • stageRewrite

        public void stageRewrite​(Table table,
                         java.lang.String fileSetId,
                         java.util.Set<F> newFiles)
        Called to persist the output of a rewrite action for a specific group. Since the write is done via a Spark Datasource, we have to propagate the result through this side-effect call.
        Parameters:
        table - table where the rewrite is occurring
        fileSetId - the id used to identify the source set of files being rewritten
        newFiles - the new files which have been written

      • fetchNewFiles

        public java.util.Set<F> fetchNewFiles​(Table table,
                                      java.lang.String fileSetId)

      • clearRewrite

        public void clearRewrite​(Table table,
                         java.lang.String fileSetId)

      • fetchSetIds

        public java.util.Set<java.lang.String> fetchSetIds​(Table table)