Package org.apache.iceberg

Class ManifestFiles

java.lang.Object

org.apache.iceberg.ManifestFiles

public class ManifestFiles
extends java.lang.Object

Method Summary

Modifier and Type	Method	Description
static ManifestFile	decode(byte[] manifestData)	Decode the binary data into a ManifestFile.
static void	dropCache(FileIO fileIO)	Drop manifest file cache object for a FileIO if exists.
static byte[]	encode(ManifestFile manifestFile)	Encode the ManifestFile to a byte array by using avro encoder.
static ManifestReader<DataFile>	read(ManifestFile manifest, FileIO io)	Returns a new ManifestReader for a ManifestFile.
static ManifestReader<DataFile>	read(ManifestFile manifest, FileIO io, java.util.Map<java.lang.Integer,PartitionSpec> specsById)	Returns a new ManifestReader for a ManifestFile.
static ManifestReader<DeleteFile>	readDeleteManifest(ManifestFile manifest, FileIO io, java.util.Map<java.lang.Integer,PartitionSpec> specsById)	Returns a new ManifestReader for a ManifestFile.
static CloseableIterable<java.lang.String>	readPaths(ManifestFile manifest, FileIO io)	Returns a CloseableIterable of file paths in the ManifestFile.
static ManifestWriter<DataFile>	write(int formatVersion, PartitionSpec spec, EncryptedOutputFile encryptedOutputFile, java.lang.Long snapshotId)	Create a new ManifestWriter for the given format version.
static ManifestWriter<DataFile>	write(int formatVersion, PartitionSpec spec, OutputFile outputFile, java.lang.Long snapshotId)	Create a new ManifestWriter for the given format version.
static ManifestWriter<DataFile>	write(PartitionSpec spec, OutputFile outputFile)	Create a new ManifestWriter.
static ManifestWriter<DeleteFile>	writeDeleteManifest(int formatVersion, PartitionSpec spec, EncryptedOutputFile outputFile, java.lang.Long snapshotId)	Create a new ManifestWriter for the given format version.
static ManifestWriter<DeleteFile>	writeDeleteManifest(int formatVersion, PartitionSpec spec, OutputFile outputFile, java.lang.Long snapshotId)	Create a new ManifestWriter for the given format version.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

dropCache

public static void dropCache(FileIO fileIO)

Drop manifest file cache object for a FileIO if exists.

readPaths

public static CloseableIterable<java.lang.String> readPaths(ManifestFile manifest,
                                                            FileIO io)

Returns a CloseableIterable of file paths in the ManifestFile.

Parameters:

manifest - a ManifestFile

io - a FileIO

Returns:

a manifest reader

read

public static ManifestReader<DataFile> read(ManifestFile manifest,
                                            FileIO io)

Returns a new ManifestReader for a ManifestFile.

Note: Callers should use read(ManifestFile, FileIO, Map) to ensure the schema used by filters is the latest table schema. This should be used only when reading a manifest without filters.

Parameters:

manifest - a ManifestFile

io - a FileIO

Returns:

a manifest reader

read

public static ManifestReader<DataFile> read(ManifestFile manifest,
                                            FileIO io,
                                            java.util.Map<java.lang.Integer,PartitionSpec> specsById)

Returns a new ManifestReader for a ManifestFile.

Parameters:

manifest - a ManifestFile

io - a FileIO

specsById - a Map from spec ID to partition spec

Returns:

a ManifestReader

write

public static ManifestWriter<DataFile> write(PartitionSpec spec,
                                             OutputFile outputFile)

Create a new ManifestWriter.

Manifests created by this writer have all entry snapshot IDs set to null. All entries will inherit the snapshot ID that will be assigned to the manifest on commit.

Parameters:

spec - PartitionSpec used to produce DataFile partition tuples

outputFile - the destination file location

Returns:

a manifest writer

write

public static ManifestWriter<DataFile> write(int formatVersion,
                                             PartitionSpec spec,
                                             OutputFile outputFile,
                                             java.lang.Long snapshotId)

Create a new ManifestWriter for the given format version.

Parameters:

formatVersion - a target format version

spec - a PartitionSpec

outputFile - an OutputFile where the manifest will be written

snapshotId - a snapshot ID for the manifest entries, or null for an inherited ID

Returns:

a manifest writer

write

public static ManifestWriter<DataFile> write(int formatVersion,
                                             PartitionSpec spec,
                                             EncryptedOutputFile encryptedOutputFile,
                                             java.lang.Long snapshotId)

Create a new ManifestWriter for the given format version.

Parameters:

formatVersion - a target format version

spec - a PartitionSpec

encryptedOutputFile - an EncryptedOutputFile where the manifest will be written

snapshotId - a snapshot ID for the manifest entries, or null for an inherited ID

Returns:

a manifest writer

readDeleteManifest

public static ManifestReader<DeleteFile> readDeleteManifest(ManifestFile manifest,
                                                            FileIO io,
                                                            java.util.Map<java.lang.Integer,PartitionSpec> specsById)

Returns a new ManifestReader for a ManifestFile.

Parameters:

manifest - a ManifestFile

io - a FileIO

specsById - a Map from spec ID to partition spec

Returns:

a ManifestReader

writeDeleteManifest

public static ManifestWriter<DeleteFile> writeDeleteManifest(int formatVersion,
                                                             PartitionSpec spec,
                                                             OutputFile outputFile,
                                                             java.lang.Long snapshotId)

Create a new ManifestWriter for the given format version.

Parameters:

formatVersion - a target format version

spec - a PartitionSpec

outputFile - an OutputFile where the manifest will be written

snapshotId - a snapshot ID for the manifest entries, or null for an inherited ID

Returns:

a manifest writer

writeDeleteManifest

public static ManifestWriter<DeleteFile> writeDeleteManifest(int formatVersion,
                                                             PartitionSpec spec,
                                                             EncryptedOutputFile outputFile,
                                                             java.lang.Long snapshotId)

Create a new ManifestWriter for the given format version.

Parameters:

formatVersion - a target format version

spec - a PartitionSpec

outputFile - an EncryptedOutputFile where the manifest will be written

snapshotId - a snapshot ID for the manifest entries, or null for an inherited ID

Returns:

a manifest writer

encode

public static byte[] encode(ManifestFile manifestFile)
                     throws java.io.IOException

Encode the ManifestFile to a byte array by using avro encoder.

Parameters:

manifestFile - a ManifestFile, which should always be a GenericManifestFile.

Returns:

the binary data.

Throws:

java.io.IOException - if encounter any IO error when encoding.

decode

public static ManifestFile decode(byte[] manifestData)
                           throws java.io.IOException

Decode the binary data into a ManifestFile.

Parameters:

manifestData - the binary data.

Returns:

a ManifestFile. To be precise, it's a GenericManifestFile which don't expose to public.

Throws:

java.io.IOException - if encounter any IO error when decoding.