Package org.apache.iceberg.azure.adlsv2

Class ADLSFileIO

java.lang.Object

org.apache.iceberg.azure.adlsv2.ADLSFileIO

All Implemented Interfaces:

Closeable, Serializable, AutoCloseable, DelegateFileIO, FileIO, SupportsBulkOperations, SupportsPrefixOperations

public class ADLSFileIO
extends Object
implements DelegateFileIO

FileIO implementation backed by Azure Data Lake Storage Gen2.

See Also:

• Serialized Form

Constructor Summary

Constructors

Constructor	Description
ADLSFileIO()	No-arg constructor to load the FileIO dynamically.

Method Summary

Modifier and Type	Method	Description
com.azure.storage.file.datalake.DataLakeFileSystemClient	client(String path)
void	deleteFile(String path)	Delete the file at the given path.
void	deleteFiles(Iterable<String> pathsToDelete)	Delete the files at the given paths.
void	deletePrefix(String prefix)	Delete all files under a prefix.
void	initialize(Map<String,String> props)	Initialize File IO from catalog properties.
Iterable<FileInfo>	listPrefix(String prefix)	Return an iterable of all files under a prefix.
InputFile	newInputFile(String path)	Get a InputFile instance to read bytes from the file at the given path.
InputFile	newInputFile(String path, long length)	Get a InputFile instance to read bytes from the file at the given path, with a known file length.
OutputFile	newOutputFile(String path)	Get a OutputFile instance to write bytes to the file at the given path.
Map<String,String>	properties()	Returns the property map used to configure this FileIO

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.iceberg.io.FileIO

close, deleteFile, deleteFile, newInputFile, newInputFile, newInputFile

Constructor Details

ADLSFileIO

public ADLSFileIO()

No-arg constructor to load the FileIO dynamically.

All fields are initialized by calling initialize(Map) later.

Method Details

newInputFile

public InputFile newInputFile(String path)

Description copied from interface: FileIO

Get a InputFile instance to read bytes from the file at the given path.

Specified by:

newInputFile in interface FileIO

newInputFile

public InputFile newInputFile(String path,
 long length)

Description copied from interface: FileIO

Get a InputFile instance to read bytes from the file at the given path, with a known file length.

Specified by:

newInputFile in interface FileIO

newOutputFile

public OutputFile newOutputFile(String path)

Description copied from interface: FileIO

Get a OutputFile instance to write bytes to the file at the given path.

Specified by:

newOutputFile in interface FileIO

deleteFile

public void deleteFile(String path)

Description copied from interface: FileIO

Delete the file at the given path.

Specified by:

deleteFile in interface FileIO

properties

public Map<String,String> properties()

Description copied from interface: FileIO

Returns the property map used to configure this FileIO

Specified by:

properties in interface FileIO

client

public com.azure.storage.file.datalake.DataLakeFileSystemClient client(String path)

initialize

public void initialize(Map<String,String> props)

Description copied from interface: FileIO

Initialize File IO from catalog properties.

Specified by:

initialize in interface FileIO

Parameters:

props - catalog properties

deleteFiles

public void deleteFiles(Iterable<String> pathsToDelete)
                 throws BulkDeletionFailureException

Description copied from interface: SupportsBulkOperations

Delete the files at the given paths.

Specified by:

deleteFiles in interface SupportsBulkOperations

Parameters:

pathsToDelete - The paths to delete

Throws:

BulkDeletionFailureException - in case of failure to delete at least 1 file

listPrefix

public Iterable<FileInfo> listPrefix(String prefix)

Description copied from interface: SupportsPrefixOperations

Return an iterable of all files under a prefix.

Hierarchical file systems (e.g. HDFS) may impose additional restrictions like the prefix must fully match a directory whereas key/value object stores may allow for arbitrary prefixes.

Specified by:

listPrefix in interface SupportsPrefixOperations

Parameters:

prefix - prefix to list

Returns:

iterable of file information

deletePrefix

public void deletePrefix(String prefix)

Description copied from interface: SupportsPrefixOperations

Delete all files under a prefix.

Hierarchical file systems (e.g. HDFS) may impose additional restrictions like the prefix must fully match a directory whereas key/value object stores may allow for arbitrary prefixes.

Specified by:

deletePrefix in interface SupportsPrefixOperations

Parameters:

prefix - prefix to delete