Package org.apache.iceberg.deletes
Class Deletes
- java.lang.Object
-
- org.apache.iceberg.deletes.Deletes
-
public class Deletes extends java.lang.Object
-
-
Method Summary
All Methods Static Methods Concrete Methods Deprecated Methods Modifier and Type Method Description static <T extends StructLike>
CloseableIterable<java.lang.Long>deletePositions(java.lang.CharSequence dataLocation, java.util.List<CloseableIterable<T>> deleteFiles)
static CloseableIterable<java.lang.Long>
deletePositions(java.lang.CharSequence dataLocation, CloseableIterable<StructLike> deleteFile)
static <T> CloseableIterable<T>
filter(CloseableIterable<T> rows, java.util.function.Function<T,StructLike> rowToDeleteKey, StructLikeSet deleteSet)
static <T> CloseableIterable<T>
filterDeleted(CloseableIterable<T> rows, java.util.function.Predicate<T> isDeleted, DeleteCounter counter)
Returns the remaining rows (the ones that are not deleted), while counting the deleted ones.static <T> CloseableIterable<T>
markDeleted(CloseableIterable<T> rows, java.util.function.Predicate<T> isDeleted, java.util.function.Consumer<T> deleteMarker)
Returns the same rows that are input, while marking the deleted ones.static <T> CloseableIterable<T>
streamingFilter(CloseableIterable<T> rows, java.util.function.Function<T,java.lang.Long> rowToPosition, CloseableIterable<java.lang.Long> posDeletes)
Deprecated.since 1.7.0, will be removed in 1.8.0.static <T> CloseableIterable<T>
streamingFilter(CloseableIterable<T> rows, java.util.function.Function<T,java.lang.Long> rowToPosition, CloseableIterable<java.lang.Long> posDeletes, DeleteCounter counter)
Deprecated.since 1.7.0, will be removed in 1.8.0.static <T> CloseableIterable<T>
streamingMarker(CloseableIterable<T> rows, java.util.function.Function<T,java.lang.Long> rowToPosition, CloseableIterable<java.lang.Long> posDeletes, java.util.function.Consumer<T> markRowDeleted)
Deprecated.since 1.7.0, will be removed in 1.8.0.static StructLikeSet
toEqualitySet(CloseableIterable<StructLike> eqDeletes, Types.StructType eqType)
static <T extends StructLike>
PositionDeleteIndextoPositionIndex(java.lang.CharSequence dataLocation, java.util.List<CloseableIterable<T>> deleteFiles)
Deprecated.since 1.7.0, will be removed in 1.8.0; use delete loaders.static <T extends StructLike>
PositionDeleteIndextoPositionIndex(java.lang.CharSequence dataLocation, java.util.List<CloseableIterable<T>> deleteFiles, java.util.concurrent.ExecutorService deleteWorkerPool)
Deprecated.since 1.7.0, will be removed in 1.8.0; use delete loaders.static <T extends StructLike>
PositionDeleteIndextoPositionIndex(java.lang.CharSequence dataLocation, CloseableIterable<T> posDeletes, DeleteFile file)
static PositionDeleteIndex
toPositionIndex(CloseableIterable<java.lang.Long> posDeletes)
static <T extends StructLike>
CharSequenceMap<PositionDeleteIndex>toPositionIndexes(CloseableIterable<T> posDeletes)
static <T extends StructLike>
CharSequenceMap<PositionDeleteIndex>toPositionIndexes(CloseableIterable<T> posDeletes, DeleteFile file)
Builds a map of position delete indexes by path.
-
-
-
Method Detail
-
filter
public static <T> CloseableIterable<T> filter(CloseableIterable<T> rows, java.util.function.Function<T,StructLike> rowToDeleteKey, StructLikeSet deleteSet)
-
markDeleted
public static <T> CloseableIterable<T> markDeleted(CloseableIterable<T> rows, java.util.function.Predicate<T> isDeleted, java.util.function.Consumer<T> deleteMarker)
Returns the same rows that are input, while marking the deleted ones.- Parameters:
rows
- the rows to processisDeleted
- a predicate that determines if a row is deleteddeleteMarker
- a function that marks a row as deleted- Returns:
- the processed rows
-
filterDeleted
public static <T> CloseableIterable<T> filterDeleted(CloseableIterable<T> rows, java.util.function.Predicate<T> isDeleted, DeleteCounter counter)
Returns the remaining rows (the ones that are not deleted), while counting the deleted ones.- Parameters:
rows
- the rows to processisDeleted
- a predicate that determines if a row is deletedcounter
- a counter that counts deleted rows- Returns:
- the processed rows
-
toEqualitySet
public static StructLikeSet toEqualitySet(CloseableIterable<StructLike> eqDeletes, Types.StructType eqType)
-
toPositionIndexes
public static <T extends StructLike> CharSequenceMap<PositionDeleteIndex> toPositionIndexes(CloseableIterable<T> posDeletes)
-
toPositionIndexes
public static <T extends StructLike> CharSequenceMap<PositionDeleteIndex> toPositionIndexes(CloseableIterable<T> posDeletes, DeleteFile file)
Builds a map of position delete indexes by path.Unlike
toPositionIndex(CharSequence, List)
, this method builds a position delete index for each referenced data file and does not filter deletes. This can be useful when the entire delete file content is needed (e.g. caching).- Parameters:
posDeletes
- position deletesfile
- the source delete file for the deletes- Returns:
- the map of position delete indexes by path
-
toPositionIndex
public static <T extends StructLike> PositionDeleteIndex toPositionIndex(java.lang.CharSequence dataLocation, CloseableIterable<T> posDeletes, DeleteFile file)
-
toPositionIndex
@Deprecated public static <T extends StructLike> PositionDeleteIndex toPositionIndex(java.lang.CharSequence dataLocation, java.util.List<CloseableIterable<T>> deleteFiles)
Deprecated.since 1.7.0, will be removed in 1.8.0; use delete loaders.
-
toPositionIndex
@Deprecated public static <T extends StructLike> PositionDeleteIndex toPositionIndex(java.lang.CharSequence dataLocation, java.util.List<CloseableIterable<T>> deleteFiles, java.util.concurrent.ExecutorService deleteWorkerPool)
Deprecated.since 1.7.0, will be removed in 1.8.0; use delete loaders.
-
toPositionIndex
public static PositionDeleteIndex toPositionIndex(CloseableIterable<java.lang.Long> posDeletes)
-
streamingFilter
@Deprecated public static <T> CloseableIterable<T> streamingFilter(CloseableIterable<T> rows, java.util.function.Function<T,java.lang.Long> rowToPosition, CloseableIterable<java.lang.Long> posDeletes)
Deprecated.since 1.7.0, will be removed in 1.8.0.
-
streamingFilter
@Deprecated public static <T> CloseableIterable<T> streamingFilter(CloseableIterable<T> rows, java.util.function.Function<T,java.lang.Long> rowToPosition, CloseableIterable<java.lang.Long> posDeletes, DeleteCounter counter)
Deprecated.since 1.7.0, will be removed in 1.8.0.
-
streamingMarker
@Deprecated public static <T> CloseableIterable<T> streamingMarker(CloseableIterable<T> rows, java.util.function.Function<T,java.lang.Long> rowToPosition, CloseableIterable<java.lang.Long> posDeletes, java.util.function.Consumer<T> markRowDeleted)
Deprecated.since 1.7.0, will be removed in 1.8.0.
-
deletePositions
public static CloseableIterable<java.lang.Long> deletePositions(java.lang.CharSequence dataLocation, CloseableIterable<StructLike> deleteFile)
-
deletePositions
public static <T extends StructLike> CloseableIterable<java.lang.Long> deletePositions(java.lang.CharSequence dataLocation, java.util.List<CloseableIterable<T>> deleteFiles)
-
-