Package org.apache.iceberg

Class RewriteTablePathUtil

java.lang.Object
org.apache.iceberg.RewriteTablePathUtil
public class RewriteTablePathUtil extends Object
Utilities for Rewrite table path action.

  • Field Details

  • Method Details

    • replacePaths

      public static TableMetadata replacePaths(TableMetadata metadata, String sourcePrefix, String targetPrefix)
      Create a new table metadata object, replacing path references
      Parameters:
      metadata - source table metadata
      sourcePrefix - source prefix that will be replaced
      targetPrefix - target prefix that will replace it
      Returns:
      copy of table metadata with paths replaced

    • rewriteManifestList

      public static RewriteTablePathUtil.RewriteResult<ManifestFile> rewriteManifestList(Snapshot snapshot, FileIO io, TableMetadata tableMetadata, Map<String,Long> rewrittenManifestLengths, String sourcePrefix, String targetPrefix, String stagingDir, String outputPath)
      Rewrite a manifest list representing a snapshot, replacing path references.
      Parameters:
      snapshot - snapshot represented by the manifest list
      io - file io
      tableMetadata - metadata of table
      rewrittenManifestLengths - byte length of each manifest rewritten by this run, keyed by source manifest path. Only these manifests are rewritten; any other manifest in the snapshot keeps the length recorded in the source table.
      sourcePrefix - source prefix that will be replaced
      targetPrefix - target prefix that will replace it
      stagingDir - staging directory
      outputPath - location to write the manifest list
      Returns:
      a copy plan for manifest files whose metadata were contained in the rewritten manifest list

    • rewriteDataManifest

      public static RewriteTablePathUtil.RewriteResult<DataFile> rewriteDataManifest(ManifestFile manifestFile, Set<Long> snapshotIds, OutputFile outputFile, FileIO io, int format, Map<Integer,PartitionSpec> specsById, String sourcePrefix, String targetPrefix) throws IOException
      Rewrite a data manifest, replacing path references.
      Parameters:
      manifestFile - source manifest file to rewrite
      snapshotIds - snapshot ids for filtering returned data manifest entries
      outputFile - output file to rewrite manifest file to
      io - file io
      format - format of the manifest file
      specsById - map of partition specs by id
      sourcePrefix - source prefix that will be replaced
      targetPrefix - target prefix that will replace it
      Returns:
      a copy plan of content files in the manifest that was rewritten, recording the rewritten manifest's byte length
      Throws:
      IOException

    • rewriteDeleteManifest

      @Deprecated public static RewriteTablePathUtil.RewriteResult<DeleteFile> rewriteDeleteManifest(ManifestFile manifestFile, Set<Long> snapshotIds, OutputFile outputFile, FileIO io, int format, Map<Integer,PartitionSpec> specsById, String sourcePrefix, String targetPrefix, String stagingLocation) throws IOException
      Deprecated.
      since 1.12.0, will be removed in 1.13.0; use the overload that accepts the map of rewritten position delete file sizes. This overload records the original file_size_in_bytes, which can be inconsistent with the rewritten file size on disk once embedded data file paths change length.
      Rewrite a delete manifest, replacing path references.
      Parameters:
      manifestFile - source delete manifest to rewrite
      snapshotIds - snapshot ids for filtering returned delete manifest entries
      outputFile - output file to rewrite manifest file to
      io - file io
      format - format of the manifest file
      specsById - map of partition specs by id
      sourcePrefix - source prefix that will be replaced
      targetPrefix - target prefix that will replace it
      stagingLocation - staging location for rewritten files (referred delete file will be rewritten here)
      Returns:
      a copy plan of content files in the manifest that was rewritten
      Throws:
      IOException

    • rewriteDeleteManifest

      public static RewriteTablePathUtil.RewriteResult<DeleteFile> rewriteDeleteManifest(ManifestFile manifestFile, Set<Long> snapshotIds, OutputFile outputFile, FileIO io, int format, Map<Integer,PartitionSpec> specsById, String sourcePrefix, String targetPrefix, String stagingLocation, Map<String,Long> rewrittenDeleteFileSizes) throws IOException
      Rewrite a delete manifest, replacing path references.

      This is a metadata-only operation: position delete file content is rewritten separately (see rewritePositionDelete(org.apache.iceberg.DeleteFile, org.apache.iceberg.io.OutputFile, org.apache.iceberg.io.FileIO, org.apache.iceberg.PartitionSpec, java.lang.String, java.lang.String, org.apache.iceberg.RewriteTablePathUtil.PositionDeleteReaderWriter)). The actual sizes of those rewritten files are supplied via rewrittenDeleteFileSizes and recorded in the manifest so that file_size_in_bytes stays consistent with the rewritten file on disk.

      Parameters:
      manifestFile - source delete manifest to rewrite
      snapshotIds - snapshot ids for filtering returned delete manifest entries
      outputFile - output file to rewrite manifest file to
      io - file io
      format - format of the manifest file
      specsById - map of partition specs by id
      sourcePrefix - source prefix that will be replaced
      targetPrefix - target prefix that will replace it
      stagingLocation - staging location for rewritten position delete files
      rewrittenDeleteFileSizes - map from source position delete file path to the actual size of the rewritten file; entries absent from the map keep their original size
      Returns:
      a copy plan of content files in the manifest that was rewritten, recording the rewritten manifest's byte length
      Throws:
      IOException

    • rewritePositionDeleteFile

      @Deprecated public static void rewritePositionDeleteFile(DeleteFile deleteFile, OutputFile outputFile, FileIO io, PartitionSpec spec, String sourcePrefix, String targetPrefix, RewriteTablePathUtil.PositionDeleteReaderWriter posDeleteReaderWriter) throws IOException
      Deprecated.
      since 1.12.0, will be removed in 1.13.0; use rewritePositionDelete(org.apache.iceberg.DeleteFile, org.apache.iceberg.io.OutputFile, org.apache.iceberg.io.FileIO, org.apache.iceberg.PartitionSpec, java.lang.String, java.lang.String, org.apache.iceberg.RewriteTablePathUtil.PositionDeleteReaderWriter) which returns the size of the rewritten file so callers can record an accurate file_size_in_bytes.
      Rewrite a position delete file, replacing path references.
      Parameters:
      deleteFile - source delete file to be rewritten
      outputFile - output file to rewrite delete file to
      io - file io
      spec - spec of delete file
      sourcePrefix - source prefix that will be replaced
      targetPrefix - target prefix to replace it
      posDeleteReaderWriter - class to read and write position delete files
      Throws:
      IOException

    • rewritePositionDelete

      public static long rewritePositionDelete(DeleteFile deleteFile, OutputFile outputFile, FileIO io, PartitionSpec spec, String sourcePrefix, String targetPrefix, RewriteTablePathUtil.PositionDeleteReaderWriter posDeleteReaderWriter) throws IOException
      Rewrite a position delete file, replacing path references, and return the size of the rewritten file.

      The size is measured from the writer after it is closed (rather than via a separate getLength()/HEAD call), so it is accurate even on file systems where the length of an in-progress write underreports. Callers record this size as file_size_in_bytes in the rewritten manifest.

      Parameters:
      deleteFile - source position delete file to rewrite
      outputFile - output file to write the rewritten delete file to
      io - file io
      spec - spec of delete file
      sourcePrefix - source prefix that will be replaced
      targetPrefix - target prefix to replace it
      posDeleteReaderWriter - class to read and write position delete files
      Returns:
      the size in bytes of the rewritten file
      Throws:
      IOException

    • newPath

      public static String newPath(String path, String sourcePrefix, String targetPrefix)
      Rewrite a path by replacing its source prefix with a target prefix.

      If the path equals the source prefix (representing a directory location), the result will be the target prefix with a trailing separator.

      Trailing separators are normalized: "/a" and "/a/" are treated as equivalent for both path and sourcePrefix.

      Parameters:
      path - absolute path to rewrite
      sourcePrefix - source prefix that will be replaced
      targetPrefix - target prefix that will replace it
      Returns:
      new path with source prefix replaced by target prefix
      Throws:
      IllegalArgumentException - if path is not under or equal to sourcePrefix

    • combinePaths

      public static String combinePaths(String absolutePath, String relativePath)
      Combine a base path and a relative path.

      If the relative path is empty, returns the absolute path unchanged. Otherwise, ensures a separator between the base and relative path.

      Parameters:
      absolutePath - the base path
      relativePath - the relative path to append (may be empty)
      Returns:
      the combined path, or absolutePath unchanged if relativePath is empty

    • fileName

      public static String fileName(String path)
      Returns the file name of a path.

    • relativize

      public static String relativize(String path, String prefix)
      Compute the relative path from a prefix to a given path.

      If the path is under the prefix, returns the portion after the prefix. If the path equals the prefix (representing the root directory itself), returns an empty string.

      Trailing separators are normalized: "/a" and "/a/" are treated as equivalent for both path and prefix. This allows flexibility when paths come from different sources that may or may not include trailing separators.

      Parameters:
      path - absolute path to relativize
      prefix - prefix path to remove
      Returns:
      relative path from prefix to path, or empty string if path equals prefix
      Throws:
      IllegalArgumentException - if path is not under or equal to prefix

    • maybeAppendFileSeparator

      public static String maybeAppendFileSeparator(String path)

    • stagingPath

      public static String stagingPath(String originalPath, String sourcePrefix, String stagingDir)
      Construct a staging path under a given staging directory, preserving relative directory structure to avoid conflicts when multiple files have the same name but different paths.
      Parameters:
      originalPath - source path
      sourcePrefix - source prefix to be replaced
      stagingDir - staging directory
      Returns:
      a staging path under the staging directory that preserves the relative path structure