Class SerializableTable
- java.lang.Object
-
- org.apache.iceberg.SerializableTable
-
- All Implemented Interfaces:
java.io.Serializable
,HasTableOperations
,Table
- Direct Known Subclasses:
SerializableTable.SerializableMetadataTable
,SerializableTableWithSize
public class SerializableTable extends java.lang.Object implements Table, HasTableOperations, java.io.Serializable
A read-only serializable table that can be sent to other nodes in a cluster.An instance of this class represents an immutable serializable copy of a table state and will not reflect any subsequent changed made to the original table.
While this class captures the metadata file location that can be used to load the complete table metadata, it directly persists the current schema, spec, sort order, table properties to avoid reading the metadata file from other nodes for frequently needed metadata.
The implementation assumes the passed instances of
FileIO
,EncryptionManager
are serializable. If you are serializing the table using a custom serialization framework like Kryo, those instances ofFileIO
,EncryptionManager
must be supported by that particular serialization framework.Note: loading the complete metadata from a large number of nodes can overwhelm the storage.
- See Also:
- Serialized Form
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static class
SerializableTable.SerializableMetadataTable
-
Constructor Summary
Constructors Modifier Constructor Description protected
SerializableTable(Table table)
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description static Table
copyOf(Table table)
Creates a read-only serializable table that can be sent to other nodes in a cluster.Snapshot
currentSnapshot()
Get the currentsnapshot
for this table, or null if there are no snapshots.EncryptionManager
encryption()
Returns anEncryptionManager
to encrypt and decrypt data files.ExpireSnapshots
expireSnapshots()
Create a newexpire API
to expire snapshots in this table and commit.java.util.List<HistoryEntry>
history()
Get the snapshot history of this table.FileIO
io()
Returns aFileIO
to read and write table data and metadata files.java.lang.String
location()
Return the table's base location.LocationProvider
locationProvider()
Returns aLocationProvider
to provide locations for new data files.ManageSnapshots
manageSnapshots()
Create a newmanage snapshots API
to manage snapshots in this table and commit.java.lang.String
name()
Return the full name for this table.AppendFiles
newAppend()
Create a newappend API
to add files to this table and commit.BatchScan
newBatchScan()
Create a newbatch scan
for this table.DeleteFiles
newDelete()
Create a newdelete API
to delete files in this table and commit.IncrementalAppendScan
newIncrementalAppendScan()
Create a newscan
for this table.IncrementalChangelogScan
newIncrementalChangelogScan()
Create a newIncrementalChangelogScan
for this table.OverwriteFiles
newOverwrite()
Create a newoverwrite API
to overwrite files by a filter expression.ReplacePartitions
newReplacePartitions()
Not recommended: Create a newreplace partitions API
to dynamically overwrite partitions in the table with new data.RewriteFiles
newRewrite()
Create a newrewrite API
to replace files in this table and commit.RowDelta
newRowDelta()
Create a newrow-level delta API
to remove or replace rows in existing data files.TableScan
newScan()
Create a newscan
for this table.protected Table
newTable(TableOperations ops, java.lang.String tableName)
Transaction
newTransaction()
Create a newtransaction API
to commit multiple table operations at once.StaticTableOperations
operations()
java.util.List<PartitionStatisticsFile>
partitionStatisticsFiles()
Returns the current partition statistics files for the table.java.util.Map<java.lang.String,java.lang.String>
properties()
Return a map of string properties for this table.void
refresh()
Refresh the current table metadata.java.util.Map<java.lang.String,SnapshotRef>
refs()
Returns the current refs for the tableReplaceSortOrder
replaceSortOrder()
Create a newReplaceSortOrder
to set the table sort order and commit the change.RewriteManifests
rewriteManifests()
Create a newrewrite manifests API
to replace manifests for this table and commit.Schema
schema()
Return theschema
for this table.java.util.Map<java.lang.Integer,Schema>
schemas()
Return a map ofschema
for this table.Snapshot
snapshot(long snapshotId)
Get thesnapshot
of this table with the given id, or null if there is no matching snapshot.java.lang.Iterable<Snapshot>
snapshots()
Get thesnapshots
of this table.SortOrder
sortOrder()
Return thesort order
for this table.java.util.Map<java.lang.Integer,SortOrder>
sortOrders()
Return a map of sort order IDs tosort orders
for this table.PartitionSpec
spec()
Return thepartition spec
for this table.java.util.Map<java.lang.Integer,PartitionSpec>
specs()
Return a map ofpartition specs
for this table.java.util.List<StatisticsFile>
statisticsFiles()
Returns the current statistics files for the tableUpdateLocation
updateLocation()
Create a newUpdateLocation
to update table location and commit the changes.UpdatePartitionStatistics
updatePartitionStatistics()
Create a newupdate partition statistics API
to add or remove partition statistics files in this table.UpdateProperties
updateProperties()
Create a newUpdateProperties
to update table properties and commit the changes.UpdateSchema
updateSchema()
Create a newUpdateSchema
to alter the columns of this table and commit the change.UpdatePartitionSpec
updateSpec()
Create a newUpdatePartitionSpec
to alter the partition spec of this table and commit the change.UpdateStatistics
updateStatistics()
Create a newupdate table statistics API
to add or remove statistics files in this table.java.util.UUID
uuid()
Returns the UUID of the table-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface org.apache.iceberg.Table
newFastAppend, snapshot
-
-
-
-
Constructor Detail
-
SerializableTable
protected SerializableTable(Table table)
-
-
Method Detail
-
copyOf
public static Table copyOf(Table table)
Creates a read-only serializable table that can be sent to other nodes in a cluster.- Parameters:
table
- the original table to copy the state from- Returns:
- a read-only serializable table reflecting the current state of the original table
-
newTable
protected Table newTable(TableOperations ops, java.lang.String tableName)
-
name
public java.lang.String name()
Description copied from interface:Table
Return the full name for this table.
-
location
public java.lang.String location()
Description copied from interface:Table
Return the table's base location.
-
properties
public java.util.Map<java.lang.String,java.lang.String> properties()
Description copied from interface:Table
Return a map of string properties for this table.- Specified by:
properties
in interfaceTable
- Returns:
- this table's properties map
-
schema
public Schema schema()
Description copied from interface:Table
Return theschema
for this table.
-
schemas
public java.util.Map<java.lang.Integer,Schema> schemas()
Description copied from interface:Table
Return a map ofschema
for this table.
-
spec
public PartitionSpec spec()
Description copied from interface:Table
Return thepartition spec
for this table.
-
specs
public java.util.Map<java.lang.Integer,PartitionSpec> specs()
Description copied from interface:Table
Return a map ofpartition specs
for this table.
-
sortOrder
public SortOrder sortOrder()
Description copied from interface:Table
Return thesort order
for this table.
-
sortOrders
public java.util.Map<java.lang.Integer,SortOrder> sortOrders()
Description copied from interface:Table
Return a map of sort order IDs tosort orders
for this table.- Specified by:
sortOrders
in interfaceTable
- Returns:
- this table's sort orders map
-
io
public FileIO io()
Description copied from interface:Table
Returns aFileIO
to read and write table data and metadata files.
-
encryption
public EncryptionManager encryption()
Description copied from interface:Table
Returns anEncryptionManager
to encrypt and decrypt data files.- Specified by:
encryption
in interfaceTable
-
locationProvider
public LocationProvider locationProvider()
Description copied from interface:Table
Returns aLocationProvider
to provide locations for new data files.- Specified by:
locationProvider
in interfaceTable
-
statisticsFiles
public java.util.List<StatisticsFile> statisticsFiles()
Description copied from interface:Table
Returns the current statistics files for the table- Specified by:
statisticsFiles
in interfaceTable
- Returns:
- the current statistics files for the table
-
partitionStatisticsFiles
public java.util.List<PartitionStatisticsFile> partitionStatisticsFiles()
Description copied from interface:Table
Returns the current partition statistics files for the table.- Specified by:
partitionStatisticsFiles
in interfaceTable
-
refs
public java.util.Map<java.lang.String,SnapshotRef> refs()
Description copied from interface:Table
Returns the current refs for the table
-
uuid
public java.util.UUID uuid()
Description copied from interface:Table
Returns the UUID of the table
-
refresh
public void refresh()
Description copied from interface:Table
Refresh the current table metadata.
-
newScan
public TableScan newScan()
Description copied from interface:Table
Create a newscan
for this table.Once a table scan is created, it can be refined to project columns and filter data.
-
newIncrementalAppendScan
public IncrementalAppendScan newIncrementalAppendScan()
Description copied from interface:Table
Create a newscan
for this table.Once a scan is created, it can be refined to project columns and filter data.
- Specified by:
newIncrementalAppendScan
in interfaceTable
- Returns:
- an incremental scan for appends only snapshots
-
newIncrementalChangelogScan
public IncrementalChangelogScan newIncrementalChangelogScan()
Description copied from interface:Table
Create a newIncrementalChangelogScan
for this table.Once a scan is created, it can be refined to project columns and filter data.
- Specified by:
newIncrementalChangelogScan
in interfaceTable
- Returns:
- an incremental changelog scan
-
newBatchScan
public BatchScan newBatchScan()
Description copied from interface:Table
Create a newbatch scan
for this table.Once a batch scan is created, it can be refined to project columns and filter data.
- Specified by:
newBatchScan
in interfaceTable
- Returns:
- a batch scan for this table
-
currentSnapshot
public Snapshot currentSnapshot()
Description copied from interface:Table
Get the currentsnapshot
for this table, or null if there are no snapshots.- Specified by:
currentSnapshot
in interfaceTable
- Returns:
- the current table Snapshot.
-
snapshot
public Snapshot snapshot(long snapshotId)
Description copied from interface:Table
Get thesnapshot
of this table with the given id, or null if there is no matching snapshot.
-
snapshots
public java.lang.Iterable<Snapshot> snapshots()
Description copied from interface:Table
Get thesnapshots
of this table.
-
history
public java.util.List<HistoryEntry> history()
Description copied from interface:Table
Get the snapshot history of this table.- Specified by:
history
in interfaceTable
- Returns:
- a list of
history entries
-
updateSchema
public UpdateSchema updateSchema()
Description copied from interface:Table
Create a newUpdateSchema
to alter the columns of this table and commit the change.- Specified by:
updateSchema
in interfaceTable
- Returns:
- a new
UpdateSchema
-
updateSpec
public UpdatePartitionSpec updateSpec()
Description copied from interface:Table
Create a newUpdatePartitionSpec
to alter the partition spec of this table and commit the change.- Specified by:
updateSpec
in interfaceTable
- Returns:
- a new
UpdatePartitionSpec
-
updateProperties
public UpdateProperties updateProperties()
Description copied from interface:Table
Create a newUpdateProperties
to update table properties and commit the changes.- Specified by:
updateProperties
in interfaceTable
- Returns:
- a new
UpdateProperties
-
replaceSortOrder
public ReplaceSortOrder replaceSortOrder()
Description copied from interface:Table
Create a newReplaceSortOrder
to set the table sort order and commit the change.- Specified by:
replaceSortOrder
in interfaceTable
- Returns:
- a new
ReplaceSortOrder
-
updateLocation
public UpdateLocation updateLocation()
Description copied from interface:Table
Create a newUpdateLocation
to update table location and commit the changes.- Specified by:
updateLocation
in interfaceTable
- Returns:
- a new
UpdateLocation
-
newAppend
public AppendFiles newAppend()
Description copied from interface:Table
Create a newappend API
to add files to this table and commit.- Specified by:
newAppend
in interfaceTable
- Returns:
- a new
AppendFiles
-
newRewrite
public RewriteFiles newRewrite()
Description copied from interface:Table
Create a newrewrite API
to replace files in this table and commit.- Specified by:
newRewrite
in interfaceTable
- Returns:
- a new
RewriteFiles
-
rewriteManifests
public RewriteManifests rewriteManifests()
Description copied from interface:Table
Create a newrewrite manifests API
to replace manifests for this table and commit.- Specified by:
rewriteManifests
in interfaceTable
- Returns:
- a new
RewriteManifests
-
newOverwrite
public OverwriteFiles newOverwrite()
Description copied from interface:Table
Create a newoverwrite API
to overwrite files by a filter expression.- Specified by:
newOverwrite
in interfaceTable
- Returns:
- a new
OverwriteFiles
-
newRowDelta
public RowDelta newRowDelta()
Description copied from interface:Table
Create a newrow-level delta API
to remove or replace rows in existing data files.- Specified by:
newRowDelta
in interfaceTable
- Returns:
- a new
RowDelta
-
newReplacePartitions
public ReplacePartitions newReplacePartitions()
Description copied from interface:Table
Not recommended: Create a newreplace partitions API
to dynamically overwrite partitions in the table with new data.This is provided to implement SQL compatible with Hive table operations but is not recommended. Instead, use the
overwrite API
to explicitly overwrite data.- Specified by:
newReplacePartitions
in interfaceTable
- Returns:
- a new
ReplacePartitions
-
newDelete
public DeleteFiles newDelete()
Description copied from interface:Table
Create a newdelete API
to delete files in this table and commit.- Specified by:
newDelete
in interfaceTable
- Returns:
- a new
DeleteFiles
-
updateStatistics
public UpdateStatistics updateStatistics()
Description copied from interface:Table
Create a newupdate table statistics API
to add or remove statistics files in this table.- Specified by:
updateStatistics
in interfaceTable
- Returns:
- a new
UpdateStatistics
-
updatePartitionStatistics
public UpdatePartitionStatistics updatePartitionStatistics()
Description copied from interface:Table
Create a newupdate partition statistics API
to add or remove partition statistics files in this table.- Specified by:
updatePartitionStatistics
in interfaceTable
- Returns:
- a new
UpdatePartitionStatistics
-
expireSnapshots
public ExpireSnapshots expireSnapshots()
Description copied from interface:Table
Create a newexpire API
to expire snapshots in this table and commit.- Specified by:
expireSnapshots
in interfaceTable
- Returns:
- a new
ExpireSnapshots
-
manageSnapshots
public ManageSnapshots manageSnapshots()
Description copied from interface:Table
Create a newmanage snapshots API
to manage snapshots in this table and commit.- Specified by:
manageSnapshots
in interfaceTable
- Returns:
- a new
ManageSnapshots
-
newTransaction
public Transaction newTransaction()
Description copied from interface:Table
Create a newtransaction API
to commit multiple table operations at once.- Specified by:
newTransaction
in interfaceTable
- Returns:
- a new
Transaction
-
operations
public StaticTableOperations operations()
- Specified by:
operations
in interfaceHasTableOperations
-
-