Package org.apache.iceberg.hadoop

Class HadoopTables

java.lang.Object
org.apache.iceberg.hadoop.HadoopTables
All Implemented Interfaces:
org.apache.hadoop.conf.Configurable, Tables
public class HadoopTables extends Object implements Tables, org.apache.hadoop.conf.Configurable
Implementation of Iceberg tables that uses the Hadoop FileSystem to store metadata and manifests.

  • Field Details

  • Constructor Details

    • HadoopTables

      public HadoopTables()

    • HadoopTables

      public HadoopTables(org.apache.hadoop.conf.Configuration conf)

  • Method Details

    • load

      public Table load(String location)
      Loads the table location from a FileSystem path location.
      Specified by:
      load in interface Tables
      Parameters:
      location - a path URI (e.g. hdfs:///warehouse/my_table/)
      Returns:
      table implementation

    • exists

      public boolean exists(String location)
      Specified by:
      exists in interface Tables

    • create

      public Table create(Schema schema, PartitionSpec spec, SortOrder order, Map<String,String> properties, String location)
      Create a table using the FileSystem implementation resolve from location.
      Specified by:
      create in interface Tables
      Parameters:
      schema - iceberg schema used to create the table
      spec - partitioning spec, if null the table will be unpartitioned
      properties - a string map of table properties, initialized to empty if null
      location - a path URI (e.g. hdfs:///warehouse/my_table)
      Returns:
      newly created table implementation

    • dropTable

      public boolean dropTable(String location)
      Drop a table and delete all data and metadata files.
      Parameters:
      location - a path URI (e.g. hdfs:///warehouse/my_table)
      Returns:
      true if the table was dropped, false if it did not exist

    • dropTable

      public boolean dropTable(String location, boolean purge)
      Drop a table; optionally delete data and metadata files.

      If purge is set to true the implementation should delete all data and metadata files.

      Parameters:
      location - a path URI (e.g. hdfs:///warehouse/my_table)
      purge - if true, delete all data and metadata files in the table
      Returns:
      true if the table was dropped, false if it did not exist

    • newCreateTableTransaction

      public Transaction newCreateTableTransaction(String location, Schema schema, PartitionSpec spec, Map<String,String> properties)
      Start a transaction to create a table.
      Parameters:
      location - a location for the table
      schema - a schema
      spec - a partition spec
      properties - a string map of table properties
      Returns:
      a Transaction to create the table
      Throws:
      AlreadyExistsException - if the table already exists

    • newReplaceTableTransaction

      public Transaction newReplaceTableTransaction(String location, Schema schema, PartitionSpec spec, Map<String,String> properties, boolean orCreate)
      Start a transaction to replace a table.
      Parameters:
      location - a location for the table
      schema - a schema
      spec - a partition spec
      properties - a string map of table properties
      orCreate - whether to create the table if not exists
      Returns:
      a Transaction to replace the table
      Throws:
      NoSuchTableException - if the table doesn't exist and orCreate is false

    • buildTable

      public Catalog.TableBuilder buildTable(String location, Schema schema)

    • setConf

      public void setConf(org.apache.hadoop.conf.Configuration conf)
      Specified by:
      setConf in interface org.apache.hadoop.conf.Configurable

    • getConf

      public org.apache.hadoop.conf.Configuration getConf()
      Specified by:
      getConf in interface org.apache.hadoop.conf.Configurable