Package org.apache.iceberg.dell.ecs

Class EcsCatalog

java.lang.Object

org.apache.iceberg.BaseMetastoreCatalog

org.apache.iceberg.dell.ecs.EcsCatalog

All Implemented Interfaces:

Closeable, AutoCloseable, Catalog, SupportsNamespaces, Configurable<Object>

public class EcsCatalog
extends BaseMetastoreCatalog
implements SupportsNamespaces, Configurable<Object>

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.iceberg.BaseMetastoreCatalog

BaseMetastoreCatalog.BaseMetastoreCatalogTableBuilder

Nested classes/interfaces inherited from interface org.apache.iceberg.catalog.Catalog

Catalog.TableBuilder

Constructor Summary

Constructors

Constructor	Description
EcsCatalog()	No-arg constructor to load the catalog dynamically.

Method Summary

Modifier and Type	Method	Description
void	close()
void	createNamespace(Namespace namespace, Map<String,String> properties)	Create a namespace in the catalog.
protected String	defaultWarehouseLocation(TableIdentifier tableIdentifier)
boolean	dropNamespace(Namespace namespace)	Drop a namespace.
boolean	dropTable(TableIdentifier identifier, boolean purge)	Remove table object.
void	initialize(String name, Map<String,String> properties)	Initialize a catalog given a custom name and a map of catalog properties.
List<Namespace>	listNamespaces(Namespace namespace)	List child namespaces from the namespace.
List<TableIdentifier>	listTables(Namespace namespace)	Iterate all table objects with the namespace prefix.
Map<String,String>	loadNamespaceMetadata(Namespace namespace)	Load namespace properties.
String	name()	Return the name for this catalog.
boolean	namespaceExists(Namespace namespace)	Checks whether the Namespace exists.
protected TableOperations	newTableOps(TableIdentifier tableIdentifier)
Optional<com.emc.object.s3.S3ObjectMetadata>	objectMetadata(org.apache.iceberg.dell.ecs.EcsURI uri)	Get S3 object metadata which include E-Tag, user metadata and so on.
protected Map<String,String>	properties()
boolean	removeProperties(Namespace namespace, Set<String> properties)	Remove a set of property keys from a namespace in the catalog.
void	renameTable(TableIdentifier from, TableIdentifier to)	Table rename will only move table object, the data objects will still be in-place.
void	setConf(Object conf)
boolean	setProperties(Namespace namespace, Map<String,String> properties)	Set a collection of properties on a namespace in the catalog.
boolean	tableExists(TableIdentifier identifier)	Check whether table exists.
boolean	updateProperties(Namespace namespace, Consumer<Map<String,String>> propertiesFn)

Methods inherited from class org.apache.iceberg.BaseMetastoreCatalog

buildTable, fullTableName, isValidIdentifier, isValidMetadataIdentifier, loadTable, metricsReporter, registerTable, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.iceberg.catalog.Catalog

createTable, createTable, createTable, createTable, dropTable, invalidateTable, newCreateTableTransaction, newCreateTableTransaction, newCreateTableTransaction, newCreateTableTransaction, newReplaceTableTransaction, newReplaceTableTransaction, newReplaceTableTransaction, newReplaceTableTransaction

Methods inherited from interface org.apache.iceberg.catalog.SupportsNamespaces

createNamespace, listNamespaces

Constructor Details

EcsCatalog

public EcsCatalog()

No-arg constructor to load the catalog dynamically.

All fields are initialized by calling initialize(String, Map) later.

Method Details

initialize

public void initialize(String name,
 Map<String,String> properties)

Description copied from interface: Catalog

Initialize a catalog given a custom name and a map of catalog properties.

A custom Catalog implementation must have a no-arg constructor. A compute engine like Spark or Flink will first initialize the catalog without any arguments, and then call this method to complete catalog initialization with properties passed into the engine.

Specified by:

initialize in interface Catalog

Parameters:

name - a custom name for the catalog

properties - catalog properties

newTableOps

protected TableOperations newTableOps(TableIdentifier tableIdentifier)

Specified by:

newTableOps in class BaseMetastoreCatalog

defaultWarehouseLocation

protected String defaultWarehouseLocation(TableIdentifier tableIdentifier)

Specified by:

defaultWarehouseLocation in class BaseMetastoreCatalog

listTables

public List<TableIdentifier> listTables(Namespace namespace)

Iterate all table objects with the namespace prefix.

Specified by:

listTables in interface Catalog

Parameters:

namespace - a namespace

Returns:

a list of identifiers for tables

dropTable

public boolean dropTable(TableIdentifier identifier,
 boolean purge)

Remove table object. If the purge flag is set, remove all data objects.

Specified by:

dropTable in interface Catalog

Parameters:

identifier - a table identifier

purge - if true, delete all data and metadata files in the table

Returns:

true if the table was dropped, false if the table did not exist

renameTable

public void renameTable(TableIdentifier from,
 TableIdentifier to)

Table rename will only move table object, the data objects will still be in-place.

Specified by:

renameTable in interface Catalog

Parameters:

from - identifier of the table to rename

to - new table name

createNamespace

public void createNamespace(Namespace namespace,
 Map<String,String> properties)

Description copied from interface: SupportsNamespaces

Create a namespace in the catalog.

Specified by:

createNamespace in interface SupportsNamespaces

Parameters:

namespace - a multi-part namespace

properties - a string Map of properties for the given namespace

listNamespaces

public List<Namespace> listNamespaces(Namespace namespace)
                               throws NoSuchNamespaceException

Description copied from interface: SupportsNamespaces

List child namespaces from the namespace.

For two existing tables named 'a.b.c.table' and 'a.b.d.table', this method returns:

• Given: Namespace.empty()
• Returns: Namespace.of("a")

• Given: Namespace.of("a")
• Returns: Namespace.of("a", "b")

• Given: Namespace.of("a", "b")
• Returns: Namespace.of("a", "b", "c") and Namespace.of("a", "b", "d")

• Given: Namespace.of("a", "b", "c")
• Returns: empty list, because there are no child namespaces

Specified by:

listNamespaces in interface SupportsNamespaces

Returns:

a List of child Namespace names from the given namespace

Throws:

NoSuchNamespaceException - If the namespace does not exist (optional)

loadNamespaceMetadata

public Map<String,String> loadNamespaceMetadata(Namespace namespace)
                                         throws NoSuchNamespaceException

Load namespace properties.

Specified by:

loadNamespaceMetadata in interface SupportsNamespaces

Parameters:

namespace - a namespace. Namespace

Returns:

a string map of properties for the given namespace

Throws:

NoSuchNamespaceException - If the namespace does not exist (optional)

dropNamespace

public boolean dropNamespace(Namespace namespace)
                      throws NamespaceNotEmptyException

Description copied from interface: SupportsNamespaces

Drop a namespace. If the namespace exists and was dropped, this will return true.

Specified by:

dropNamespace in interface SupportsNamespaces

Parameters:

namespace - a namespace. Namespace

Returns:

true if the namespace was dropped, false otherwise.

Throws:

NamespaceNotEmptyException - If the namespace is not empty

setProperties

public boolean setProperties(Namespace namespace,
 Map<String,String> properties)
                      throws NoSuchNamespaceException

Description copied from interface: SupportsNamespaces

Set a collection of properties on a namespace in the catalog.

Properties that are not in the given map are not modified or removed by this method.

Specified by:

setProperties in interface SupportsNamespaces

Parameters:

namespace - a namespace. Namespace

properties - a collection of metadata to apply to the namespace

Throws:

NoSuchNamespaceException - If the namespace does not exist (optional)

removeProperties

public boolean removeProperties(Namespace namespace,
 Set<String> properties)
                         throws NoSuchNamespaceException

Description copied from interface: SupportsNamespaces

Remove a set of property keys from a namespace in the catalog.

Properties that are not in the given set are not modified or removed by this method.

Specified by:

removeProperties in interface SupportsNamespaces

Parameters:

namespace - a namespace. Namespace

properties - a collection of metadata to apply to the namespace

Throws:

NoSuchNamespaceException - If the namespace does not exist (optional)

updateProperties

public boolean updateProperties(Namespace namespace,
 Consumer<Map<String,String>> propertiesFn)
                         throws NoSuchNamespaceException

Throws:

NoSuchNamespaceException

namespaceExists

public boolean namespaceExists(Namespace namespace)

Description copied from interface: SupportsNamespaces

Checks whether the Namespace exists.

Specified by:

namespaceExists in interface SupportsNamespaces

Parameters:

namespace - a namespace. Namespace

Returns:

true if the Namespace exists, false otherwise

tableExists

public boolean tableExists(TableIdentifier identifier)

Description copied from interface: Catalog

Check whether table exists.

Specified by:

tableExists in interface Catalog

Parameters:

identifier - a table identifier

Returns:

true if the table exists, false otherwise

objectMetadata

public Optional<com.emc.object.s3.S3ObjectMetadata> objectMetadata(org.apache.iceberg.dell.ecs.EcsURI uri)

Get S3 object metadata which include E-Tag, user metadata and so on.

name

public String name()

Description copied from interface: Catalog

Return the name for this catalog.

Specified by:

name in interface Catalog

Returns:

this catalog's name

close

public void close()
           throws IOException

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

Overrides:

close in class BaseMetastoreCatalog

Throws:

IOException

setConf

public void setConf(Object conf)

Specified by:

setConf in interface Configurable<Object>

properties

protected Map<String,String> properties()

Overrides:

properties in class BaseMetastoreCatalog