Class HadoopCatalog

    • Constructor Detail

      • HadoopCatalog

        public HadoopCatalog()
      • HadoopCatalog

        @Deprecated
        public HadoopCatalog​(org.apache.hadoop.conf.Configuration conf,
                             java.lang.String warehouseLocation)
        Deprecated.
        please use the no-arg constructor, setConf and initialize to construct the catalog. Will be removed in v0.13.0
        The constructor of the HadoopCatalog. It uses the passed location as its warehouse directory.
        Parameters:
        conf - The Hadoop configuration
        warehouseLocation - The location used as warehouse directory
      • HadoopCatalog

        @Deprecated
        public HadoopCatalog​(org.apache.hadoop.conf.Configuration conf)
        Deprecated.
        please use the no-arg constructor, setConf and initialize to construct the catalog. Will be removed in v0.13.0
        The constructor of the HadoopCatalog. It gets the value of fs.defaultFS property from the passed Hadoop configuration as its default file system, and use the default directory iceberg/warehouse as the warehouse directory.
        Parameters:
        conf - The Hadoop configuration
    • Method Detail

      • initialize

        public void initialize​(java.lang.String name,
                               java.util.Map<java.lang.String,​java.lang.String> properties)
        Description copied from interface: Catalog
        Initialize a catalog given a custom name and a map of catalog properties.

        A custom Catalog implementation must have a no-arg constructor. A compute engine like Spark or Flink will first initialize the catalog without any arguments, and then call this method to complete catalog initialization with properties passed into the engine.

        Specified by:
        initialize in interface Catalog
        Parameters:
        name - a custom name for the catalog
        properties - catalog properties
      • name

        public java.lang.String name()
        Description copied from interface: Catalog
        Return the name for this catalog.
        Specified by:
        name in interface Catalog
        Returns:
        this catalog's name
      • listTables

        public java.util.List<TableIdentifier> listTables​(Namespace namespace)
        Description copied from interface: Catalog
        Return all the identifiers under this namespace.
        Specified by:
        listTables in interface Catalog
        Parameters:
        namespace - a namespace
        Returns:
        a list of identifiers for tables
      • dropTable

        public boolean dropTable​(TableIdentifier identifier,
                                 boolean purge)
        Description copied from interface: Catalog
        Drop a table; optionally delete data and metadata files.

        If purge is set to true the implementation should delete all data and metadata files.

        Specified by:
        dropTable in interface Catalog
        Parameters:
        identifier - a table identifier
        purge - if true, delete all data and metadata files in the table
        Returns:
        true if the table was dropped, false if the table did not exist
      • createNamespace

        public void createNamespace​(Namespace namespace,
                                    java.util.Map<java.lang.String,​java.lang.String> meta)
        Description copied from interface: SupportsNamespaces
        Create a namespace in the catalog.
        Specified by:
        createNamespace in interface SupportsNamespaces
        Parameters:
        namespace - a multi-part namespace
        meta - a string Map of properties for the given namespace
      • dropNamespace

        public boolean dropNamespace​(Namespace namespace)
        Description copied from interface: SupportsNamespaces
        Drop a namespace. If the namespace exists and was dropped, this will return true.
        Specified by:
        dropNamespace in interface SupportsNamespaces
        Parameters:
        namespace - a namespace. Namespace
        Returns:
        true if the namespace was dropped, false otherwise.
      • setProperties

        public boolean setProperties​(Namespace namespace,
                                     java.util.Map<java.lang.String,​java.lang.String> properties)
        Description copied from interface: SupportsNamespaces
        Apply a set of metadata to a namespace in the catalog.
        Specified by:
        setProperties in interface SupportsNamespaces
        Parameters:
        namespace - a namespace. Namespace
        properties - a collection of metadata to apply to the namespace
      • removeProperties

        public boolean removeProperties​(Namespace namespace,
                                        java.util.Set<java.lang.String> properties)
        Description copied from interface: SupportsNamespaces
        Remove a set of metadata from a namespace in the catalog.
        Specified by:
        removeProperties in interface SupportsNamespaces
        Parameters:
        namespace - a namespace. Namespace
        properties - a collection of metadata to apply to the namespace
      • loadNamespaceMetadata

        public java.util.Map<java.lang.String,​java.lang.String> loadNamespaceMetadata​(Namespace namespace)
        Description copied from interface: SupportsNamespaces
        Load metadata properties for a namespace.
        Specified by:
        loadNamespaceMetadata in interface SupportsNamespaces
        Parameters:
        namespace - a namespace. Namespace
        Returns:
        a string map of properties for the given namespace
      • close

        public void close()
                   throws java.io.IOException
        Specified by:
        close in interface java.lang.AutoCloseable
        Specified by:
        close in interface java.io.Closeable
        Throws:
        java.io.IOException
      • setConf

        public void setConf​(org.apache.hadoop.conf.Configuration conf)
        Specified by:
        setConf in interface org.apache.hadoop.conf.Configurable
      • getConf

        public org.apache.hadoop.conf.Configuration getConf()
        Specified by:
        getConf in interface org.apache.hadoop.conf.Configurable