Package org.apache.iceberg.aws.s3

Class S3FileIO

  • All Implemented Interfaces:
    java.io.Serializable, FileIO
    public class S3FileIO
extends java.lang.Object
implements FileIO
    FileIO implementation backed by S3.

    Locations used must follow the conventions for S3 URIs (e.g. s3://bucket/path...). URIs with schemes s3a, s3n, https are also treated as s3 file paths. Using this FileIO with other schemes will result in ValidationException.

    See Also:
    Serialized Form

    • Constructor Summary

      Constructors 
      Constructor Description
      S3FileIO()
      No-arg constructor to load the FileIO dynamically.
      S3FileIO​(SerializableSupplier<software.amazon.awssdk.services.s3.S3Client> s3)
      Constructor with custom s3 supplier and default AWS properties.
      S3FileIO​(SerializableSupplier<software.amazon.awssdk.services.s3.S3Client> s3, AwsProperties awsProperties)
      Constructor with custom s3 supplier and AWS properties.

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void deleteFile​(java.lang.String path)
      Delete the file at the given path.
      void initialize​(java.util.Map<java.lang.String,​java.lang.String> properties)
      Initialize File IO from catalog properties.
      InputFile newInputFile​(java.lang.String path)
      Get a InputFile instance to read bytes from the file at the given path.
      OutputFile newOutputFile​(java.lang.String path)
      Get a OutputFile instance to write bytes to the file at the given path.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Constructor Detail

      • S3FileIO

        public S3FileIO()
        No-arg constructor to load the FileIO dynamically.

        All fields are initialized by calling initialize(Map) later.

      • S3FileIO

        public S3FileIO​(SerializableSupplier<software.amazon.awssdk.services.s3.S3Client> s3)
        Constructor with custom s3 supplier and default AWS properties.

        Calling initialize(Map) will overwrite information set in this constructor.

        Parameters:
        s3 - s3 supplier

      • S3FileIO

        public S3FileIO​(SerializableSupplier<software.amazon.awssdk.services.s3.S3Client> s3,
                AwsProperties awsProperties)
        Constructor with custom s3 supplier and AWS properties.

        Calling initialize(Map) will overwrite information set in this constructor.

        Parameters:
        s3 - s3 supplier
        awsProperties - aws properties

    • Method Detail

      • newInputFile

        public InputFile newInputFile​(java.lang.String path)
        Description copied from interface: FileIO
        Get a InputFile instance to read bytes from the file at the given path.
        Specified by:
        newInputFile in interface FileIO

      • newOutputFile

        public OutputFile newOutputFile​(java.lang.String path)
        Description copied from interface: FileIO
        Get a OutputFile instance to write bytes to the file at the given path.
        Specified by:
        newOutputFile in interface FileIO

      • deleteFile

        public void deleteFile​(java.lang.String path)
        Description copied from interface: FileIO
        Delete the file at the given path.
        Specified by:
        deleteFile in interface FileIO

      • initialize

        public void initialize​(java.util.Map<java.lang.String,​java.lang.String> properties)
        Description copied from interface: FileIO
        Initialize File IO from catalog properties.
        Specified by:
        initialize in interface FileIO
        Parameters:
        properties - catalog properties