Package org.apache.iceberg.hive

Class HiveTableOperations

java.lang.Object
org.apache.iceberg.BaseMetastoreOperations
org.apache.iceberg.BaseMetastoreTableOperations
org.apache.iceberg.hive.HiveTableOperations
All Implemented Interfaces:
TableOperations
public class HiveTableOperations extends BaseMetastoreTableOperations
TODO we should be able to extract some more commonalities to BaseMetastoreTableOperations to avoid code duplication between this class and Metacat Tables.

  • Field Details

  • Constructor Details

    • HiveTableOperations

      protected HiveTableOperations(org.apache.hadoop.conf.Configuration conf, ClientPool<org.apache.hadoop.hive.metastore.IMetaStoreClient,org.apache.thrift.TException> metaClients, FileIO fileIO, String catalogName, String database, String table)

  • Method Details

    • translateToIcebergProp

      public static String translateToIcebergProp(String hmsProp)
      Provides key translation where necessary between Iceberg and HMS props. This translation is needed because some properties control the same behaviour but are named differently in Iceberg and Hive. Therefore changes to these property pairs should be synchronized.

      Example: Deleting data files upon DROP TABLE is enabled using gc.enabled=true in Iceberg and external.table.purge=true in Hive. Hive and Iceberg users are unaware of each other's control flags, therefore inconsistent behaviour can occur from e.g. a Hive user's point of view if external.table.purge=true is set on the HMS table but gc.enabled=false is set on the Iceberg table, resulting in no data file deletion.

      Parameters:
      hmsProp - The HMS property that should be translated to Iceberg property
      Returns:
      Iceberg property equivalent to the hmsProp. If no such translation exists, the original hmsProp is returned

    • tableName

      protected String tableName()
      Description copied from class: BaseMetastoreTableOperations
      The full name of the table used for logging purposes only. For example for HiveTableOperations it is catalogName + "." + database + "." + table.
      Specified by:
      tableName in class BaseMetastoreTableOperations
      Returns:
      The full name

    • io

      public FileIO io()
      Description copied from interface: TableOperations
      Returns a FileIO to read and write table data and metadata files.
      Specified by:
      io in interface TableOperations

    • doRefresh

      protected void doRefresh()
      Overrides:
      doRefresh in class BaseMetastoreTableOperations

    • doCommit

      protected void doCommit(TableMetadata base, TableMetadata metadata)
      Overrides:
      doCommit in class BaseMetastoreTableOperations

    • maxHiveTablePropertySize

      public long maxHiveTablePropertySize()

    • database

      public String database()

    • table

      public String table()

    • tableType

      public org.apache.hadoop.hive.metastore.TableType tableType()

    • metaClients

      public ClientPool<org.apache.hadoop.hive.metastore.IMetaStoreClient,org.apache.thrift.TException> metaClients()

    • loadHmsTable

      default org.apache.hadoop.hive.metastore.api.Table loadHmsTable() throws org.apache.thrift.TException, InterruptedException
      Throws:
      org.apache.thrift.TException
      InterruptedException

    • hmsEnvContext

      default Map<String,String> hmsEnvContext(String metadataLocation)

    • exposeInHmsProperties

      default boolean exposeInHmsProperties()

    • setSchema

      default void setSchema(Schema schema, Map<String,String> parameters)

    • setField

      default void setField(Map<String,String> parameters, String key, String value)

    • persistTable

      default void persistTable(org.apache.hadoop.hive.metastore.api.Table hmsTable, boolean updateHiveTable, String metadataLocation) throws org.apache.thrift.TException, InterruptedException
      Throws:
      org.apache.thrift.TException
      InterruptedException

    • newHmsTable

      default org.apache.hadoop.hive.metastore.api.Table newHmsTable(String hmsTableOwner)