Package org.apache.iceberg

Class BaseMetastoreTableOperations

    • Field Detail

      • ICEBERG_TABLE_TYPE_VALUE

        public static final java.lang.String ICEBERG_TABLE_TYPE_VALUE
        See Also:
        Constant Field Values

      • METADATA_LOCATION_PROP

        public static final java.lang.String METADATA_LOCATION_PROP
        See Also:
        Constant Field Values

      • PREVIOUS_METADATA_LOCATION_PROP

        public static final java.lang.String PREVIOUS_METADATA_LOCATION_PROP
        See Also:
        Constant Field Values

    • Constructor Detail

      • BaseMetastoreTableOperations

        protected BaseMetastoreTableOperations()

    • Method Detail

      • tableName

        protected abstract java.lang.String tableName()
        The full name of the table used for logging purposes only. For example for HiveTableOperations it is catalogName + "." + database + "." + table.
        Returns:
        The full name

      • currentMetadataLocation

        public java.lang.String currentMetadataLocation()

      • currentVersion

        public int currentVersion()

      • doRefresh

        protected void doRefresh()

      • commit

        public void commit​(TableMetadata base,
                   TableMetadata metadata)
        Description copied from interface: TableOperations
        Replace the base table metadata with a new version.

        This method should implement and document atomicity guarantees.

        Implementations must check that the base metadata is current to avoid overwriting updates. Once the atomic commit operation succeeds, implementations must not perform any operations that may fail because failure in this method cannot be distinguished from commit failure.

        Implementations must throw a CommitStateUnknownException in cases where it cannot be determined if the commit succeeded or failed. For example if a network partition causes the confirmation of the commit to be lost, the implementation should throw a CommitStateUnknownException. This is important because downstream users of this API need to know whether they can clean up the commit or not, if the state is unknown then it is not safe to remove any files. All other exceptions will be treated as if the commit has failed.

        Specified by:
        commit in interface TableOperations
        Parameters:
        base - table metadata on which changes were based
        metadata - new table metadata with updates

      • requestRefresh

        protected void requestRefresh()

      • disableRefresh

        protected void disableRefresh()

      • writeNewMetadataIfRequired

        protected java.lang.String writeNewMetadataIfRequired​(boolean newTable,
                                                      TableMetadata metadata)

      • writeNewMetadata

        protected java.lang.String writeNewMetadata​(TableMetadata metadata,
                                            int newVersion)

      • refreshFromMetadataLocation

        protected void refreshFromMetadataLocation​(java.lang.String newLocation)

      • refreshFromMetadataLocation

        protected void refreshFromMetadataLocation​(java.lang.String newLocation,
                                           int numRetries)

      • refreshFromMetadataLocation

        protected void refreshFromMetadataLocation​(java.lang.String newLocation,
                                           java.util.function.Predicate<java.lang.Exception> shouldRetry,
                                           int numRetries)

      • refreshFromMetadataLocation

        protected void refreshFromMetadataLocation​(java.lang.String newLocation,
                                           java.util.function.Predicate<java.lang.Exception> shouldRetry,
                                           int numRetries,
                                           java.util.function.Function<java.lang.String,​TableMetadata> metadataLoader)

      • metadataFileLocation

        public java.lang.String metadataFileLocation​(java.lang.String filename)
        Description copied from interface: TableOperations
        Given the name of a metadata file, obtain the full path of that file using an appropriate base location of the implementation's choosing.

        The file may not exist yet, in which case the path should be returned as if it were to be created by e.g. FileIO.newOutputFile(String).

        Specified by:
        metadataFileLocation in interface TableOperations

      • checkCommitStatus

        protected BaseMetastoreOperations.CommitStatus checkCommitStatus​(java.lang.String newMetadataLocation,
                                                                 TableMetadata config)
        Attempt to load the table and see if any current or past metadata location matches the one we were attempting to set. This is used as a last resort when we are dealing with exceptions that may indicate the commit has failed but are not proof that this is the case. Past locations must also be searched on the chance that a second committer was able to successfully commit on top of our commit.
        Parameters:
        newMetadataLocation - the path of the new commit file
        config - metadata to use for configuration
        Returns:
        Commit Status of Success, Failure or Unknown