Class GlueCatalog

    • Constructor Detail

      • GlueCatalog

        public GlueCatalog()
        No-arg constructor to load the catalog dynamically.

        All fields are initialized by calling initialize(String, Map) later.

    • Method Detail

      • initialize

        public void initialize​(java.lang.String name,
                               java.util.Map<java.lang.String,​java.lang.String> properties)
        Description copied from interface: Catalog
        Initialize a catalog given a custom name and a map of catalog properties.

        A custom Catalog implementation must have a no-arg constructor. A compute engine like Spark or Flink will first initialize the catalog without any arguments, and then call this method to complete catalog initialization with properties passed into the engine.

        Specified by:
        initialize in interface Catalog
        name - a custom name for the catalog
        properties - catalog properties
      • defaultWarehouseLocation

        protected java.lang.String defaultWarehouseLocation​(TableIdentifier tableIdentifier)
        This method produces the same result as using a HiveCatalog. If databaseUri exists for the Glue database URI, the default location is databaseUri/tableName. If not, the default location is warehousePath/databaseName.db/tableName
        Specified by:
        defaultWarehouseLocation in class BaseMetastoreCatalog
        tableIdentifier - table id
        default warehouse path
      • listTables

        public java.util.List<TableIdentifier> listTables​(Namespace namespace)
        Description copied from interface: Catalog
        Return all the identifiers under this namespace.
        Specified by:
        listTables in interface Catalog
        namespace - a namespace
        a list of identifiers for tables
      • dropTable

        public boolean dropTable​(TableIdentifier identifier,
                                 boolean purge)
        Description copied from interface: Catalog
        Drop a table; optionally delete data and metadata files.

        If purge is set to true the implementation should delete all data and metadata files.

        Specified by:
        dropTable in interface Catalog
        identifier - a table identifier
        purge - if true, delete all data and metadata files in the table
        true if the table was dropped, false if the table did not exist
      • renameTable

        public void renameTable​(TableIdentifier from,
                                TableIdentifier to)
        Rename table in Glue is a drop table and create table.
        Specified by:
        renameTable in interface Catalog
        from - identifier of the table to rename
        to - new table name
      • createNamespace

        public void createNamespace​(Namespace namespace,
                                    java.util.Map<java.lang.String,​java.lang.String> metadata)
        Description copied from interface: SupportsNamespaces
        Create a namespace in the catalog.
        Specified by:
        createNamespace in interface SupportsNamespaces
        namespace - a multi-part namespace
        metadata - a string Map of properties for the given namespace
      • name

        public java.lang.String name()
        Description copied from interface: Catalog
        Return the name for this catalog.
        Specified by:
        name in interface Catalog
        this catalog's name
      • close

        public void close()
        Specified by:
        close in interface java.lang.AutoCloseable
        Specified by:
        close in interface
      • setConf

        public void setConf​(org.apache.hadoop.conf.Configuration conf)
        Specified by:
        setConf in interface org.apache.hadoop.conf.Configurable
      • getConf

        public org.apache.hadoop.conf.Configuration getConf()
        Specified by:
        getConf in interface org.apache.hadoop.conf.Configurable