Class SerializableTable
- All Implemented Interfaces:
Serializable,HasTableOperations,Table
- Direct Known Subclasses:
SerializableTable.SerializableMetadataTable,SerializableTableWithSize
An instance of this class represents an immutable serializable copy of a table state and will not reflect any subsequent changed made to the original table.
While this class captures the metadata file location that can be used to load the complete table metadata, it directly persists the current schema, spec, sort order, table properties to avoid reading the metadata file from other nodes for frequently needed metadata.
The implementation assumes the passed instances of FileIO, EncryptionManager
are serializable. If you are serializing the table using a custom serialization framework like
Kryo, those instances of FileIO, EncryptionManager must be supported by that
particular serialization framework.
Note: loading the complete metadata from a large number of nodes can overwhelm the storage.
- See Also:
-
Nested Class Summary
Nested Classes -
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionstatic TableCreates a read-only serializable table that can be sent to other nodes in a cluster.Get the currentsnapshotfor this table, or null if there are no snapshots.Returns anEncryptionManagerto encrypt and decrypt data files.Create a newexpire APIto expire snapshots in this table and commit.inthistory()Get the snapshot history of this table.io()Returns aFileIOto read and write table data and metadata files.location()Return the table's base location.Returns aLocationProviderto provide locations for new data files.Create a newmanage snapshots APIto manage snapshots in this table and commit.name()Return the full name for this table.Create a newappend APIto add files to this table and commit.Create a newbatch scanfor this table.Create a newdelete APIto delete files in this table and commit.Create a newscanfor this table.Create a newIncrementalChangelogScanfor this table.Create a newoverwrite APIto overwrite files by a filter expression.Not recommended: Create a newreplace partitions APIto dynamically overwrite partitions in the table with new data.Create a newrewrite APIto replace files in this table and commit.Create a newrow-level delta APIto remove or replace rows in existing data files.newScan()Create a newscanfor this table.protected TablenewTable(TableOperations ops, String tableName) Create a newtransaction APIto commit multiple table operations at once.Returns the current partition statistics files for the table.Return a map of string properties for this table.voidrefresh()Refresh the current table metadata.refs()Returns the current refs for the tableCreate a newReplaceSortOrderto set the table sort order and commit the change.Create a newrewrite manifests APIto replace manifests for this table and commit.schema()Return theschemafor this table.schemas()Return a map ofschemafor this table.snapshot(long snapshotId) Get thesnapshotof this table with the given id, or null if there is no matching snapshot.Get thesnapshotsof this table.Return thesort orderfor this table.Return a map of sort order IDs tosort ordersfor this table.spec()Return thepartition specfor this table.specs()Return a map ofpartition specsfor this table.Returns the current statistics files for the tableCreate a newUpdateLocationto update table location and commit the changes.Create a newupdate partition statistics APIto add or remove partition statistics files in this table.Create a newUpdatePropertiesto update table properties and commit the changes.Create a newUpdateSchemato alter the columns of this table and commit the change.Create a newUpdatePartitionSpecto alter the partition spec of this table and commit the change.Create a newupdate table statistics APIto add or remove statistics files in this table.uuid()Returns the UUID of the tableMethods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, waitMethods inherited from interface org.apache.iceberg.Table
newFastAppend, snapshot
-
Constructor Details
-
SerializableTable
-
-
Method Details
-
copyOf
Creates a read-only serializable table that can be sent to other nodes in a cluster.- Parameters:
table- the original table to copy the state from- Returns:
- a read-only serializable table reflecting the current state of the original table
-
metadataFileLocation
-
newTable
-
name
Description copied from interface:TableReturn the full name for this table. -
location
Description copied from interface:TableReturn the table's base location. -
properties
Description copied from interface:TableReturn a map of string properties for this table.- Specified by:
propertiesin interfaceTable- Returns:
- this table's properties map
-
formatVersion
public int formatVersion() -
schema
Description copied from interface:TableReturn theschemafor this table. -
schemas
Description copied from interface:TableReturn a map ofschemafor this table. -
spec
Description copied from interface:TableReturn thepartition specfor this table. -
specs
Description copied from interface:TableReturn a map ofpartition specsfor this table. -
sortOrder
Description copied from interface:TableReturn thesort orderfor this table. -
sortOrders
Description copied from interface:TableReturn a map of sort order IDs tosort ordersfor this table.- Specified by:
sortOrdersin interfaceTable- Returns:
- this table's sort orders map
-
io
Description copied from interface:TableReturns aFileIOto read and write table data and metadata files. -
encryption
Description copied from interface:TableReturns anEncryptionManagerto encrypt and decrypt data files.- Specified by:
encryptionin interfaceTable
-
locationProvider
Description copied from interface:TableReturns aLocationProviderto provide locations for new data files.- Specified by:
locationProviderin interfaceTable
-
statisticsFiles
Description copied from interface:TableReturns the current statistics files for the table- Specified by:
statisticsFilesin interfaceTable- Returns:
- the current statistics files for the table
-
partitionStatisticsFiles
Description copied from interface:TableReturns the current partition statistics files for the table.- Specified by:
partitionStatisticsFilesin interfaceTable
-
refs
Description copied from interface:TableReturns the current refs for the table -
uuid
Description copied from interface:TableReturns the UUID of the table -
refresh
public void refresh()Description copied from interface:TableRefresh the current table metadata. -
newScan
Description copied from interface:TableCreate a newscanfor this table.Once a table scan is created, it can be refined to project columns and filter data.
-
newIncrementalAppendScan
Description copied from interface:TableCreate a newscanfor this table.Once a scan is created, it can be refined to project columns and filter data.
- Specified by:
newIncrementalAppendScanin interfaceTable- Returns:
- an incremental scan for appends only snapshots
-
newIncrementalChangelogScan
Description copied from interface:TableCreate a newIncrementalChangelogScanfor this table.Once a scan is created, it can be refined to project columns and filter data.
- Specified by:
newIncrementalChangelogScanin interfaceTable- Returns:
- an incremental changelog scan
-
newBatchScan
Description copied from interface:TableCreate a newbatch scanfor this table.Once a batch scan is created, it can be refined to project columns and filter data.
- Specified by:
newBatchScanin interfaceTable- Returns:
- a batch scan for this table
-
currentSnapshot
Description copied from interface:TableGet the currentsnapshotfor this table, or null if there are no snapshots.- Specified by:
currentSnapshotin interfaceTable- Returns:
- the current table Snapshot.
-
snapshot
Description copied from interface:TableGet thesnapshotof this table with the given id, or null if there is no matching snapshot. -
snapshots
Description copied from interface:TableGet thesnapshotsof this table. -
history
Description copied from interface:TableGet the snapshot history of this table.- Specified by:
historyin interfaceTable- Returns:
- a list of
history entries
-
updateSchema
Description copied from interface:TableCreate a newUpdateSchemato alter the columns of this table and commit the change.- Specified by:
updateSchemain interfaceTable- Returns:
- a new
UpdateSchema
-
updateSpec
Description copied from interface:TableCreate a newUpdatePartitionSpecto alter the partition spec of this table and commit the change.- Specified by:
updateSpecin interfaceTable- Returns:
- a new
UpdatePartitionSpec
-
updateProperties
Description copied from interface:TableCreate a newUpdatePropertiesto update table properties and commit the changes.- Specified by:
updatePropertiesin interfaceTable- Returns:
- a new
UpdateProperties
-
replaceSortOrder
Description copied from interface:TableCreate a newReplaceSortOrderto set the table sort order and commit the change.- Specified by:
replaceSortOrderin interfaceTable- Returns:
- a new
ReplaceSortOrder
-
updateLocation
Description copied from interface:TableCreate a newUpdateLocationto update table location and commit the changes.- Specified by:
updateLocationin interfaceTable- Returns:
- a new
UpdateLocation
-
newAppend
Description copied from interface:TableCreate a newappend APIto add files to this table and commit.- Specified by:
newAppendin interfaceTable- Returns:
- a new
AppendFiles
-
newRewrite
Description copied from interface:TableCreate a newrewrite APIto replace files in this table and commit.- Specified by:
newRewritein interfaceTable- Returns:
- a new
RewriteFiles
-
rewriteManifests
Description copied from interface:TableCreate a newrewrite manifests APIto replace manifests for this table and commit.- Specified by:
rewriteManifestsin interfaceTable- Returns:
- a new
RewriteManifests
-
newOverwrite
Description copied from interface:TableCreate a newoverwrite APIto overwrite files by a filter expression.- Specified by:
newOverwritein interfaceTable- Returns:
- a new
OverwriteFiles
-
newRowDelta
Description copied from interface:TableCreate a newrow-level delta APIto remove or replace rows in existing data files.- Specified by:
newRowDeltain interfaceTable- Returns:
- a new
RowDelta
-
newReplacePartitions
Description copied from interface:TableNot recommended: Create a newreplace partitions APIto dynamically overwrite partitions in the table with new data.This is provided to implement SQL compatible with Hive table operations but is not recommended. Instead, use the
overwrite APIto explicitly overwrite data.- Specified by:
newReplacePartitionsin interfaceTable- Returns:
- a new
ReplacePartitions
-
newDelete
Description copied from interface:TableCreate a newdelete APIto delete files in this table and commit.- Specified by:
newDeletein interfaceTable- Returns:
- a new
DeleteFiles
-
updateStatistics
Description copied from interface:TableCreate a newupdate table statistics APIto add or remove statistics files in this table.- Specified by:
updateStatisticsin interfaceTable- Returns:
- a new
UpdateStatistics
-
updatePartitionStatistics
Description copied from interface:TableCreate a newupdate partition statistics APIto add or remove partition statistics files in this table.- Specified by:
updatePartitionStatisticsin interfaceTable- Returns:
- a new
UpdatePartitionStatistics
-
expireSnapshots
Description copied from interface:TableCreate a newexpire APIto expire snapshots in this table and commit.- Specified by:
expireSnapshotsin interfaceTable- Returns:
- a new
ExpireSnapshots
-
manageSnapshots
Description copied from interface:TableCreate a newmanage snapshots APIto manage snapshots in this table and commit.- Specified by:
manageSnapshotsin interfaceTable- Returns:
- a new
ManageSnapshots
-
newTransaction
Description copied from interface:TableCreate a newtransaction APIto commit multiple table operations at once.- Specified by:
newTransactionin interfaceTable- Returns:
- a new
Transaction
-
operations
- Specified by:
operationsin interfaceHasTableOperations
-