org.apache.iceberg.spark

Class RemoveNetCarryoverIterator

java.lang.Object

org.apache.iceberg.spark.ChangelogIterator

org.apache.iceberg.spark.RemoveNetCarryoverIterator

All Implemented Interfaces:

java.util.Iterator<org.apache.spark.sql.Row>

public class RemoveNetCarryoverIterator
extends ChangelogIterator

This class computes the net changes across multiple snapshots. It is different from RemoveCarryoverIterator, which only removes carry-over rows within a single snapshot. It takes a row iterator, and assumes the following:

• The row iterator is partitioned by all columns.
• The row iterator is sorted by all columns, change order, and change type. The change order is 1-to-1 mapping to snapshot id.

Field Summary

Fields inherited from class org.apache.iceberg.spark.ChangelogIterator

DELETE, INSERT, UPDATE_AFTER, UPDATE_BEFORE

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	RemoveNetCarryoverIterator(java.util.Iterator<org.apache.spark.sql.Row> rowIterator, org.apache.spark.sql.types.StructType rowType)

Method Summary

Modifier and Type	Method and Description
boolean	hasNext()
org.apache.spark.sql.Row	next()

Methods inherited from class org.apache.iceberg.spark.ChangelogIterator

changeType, changeTypeIndex, computeUpdates, generateIndicesToIdentifySameRow, isDifferentValue, isSameRecord, removeCarryovers, removeNetCarryovers, rowIterator, rowType

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.util.Iterator

forEachRemaining, remove

Constructor Detail

RemoveNetCarryoverIterator

protected RemoveNetCarryoverIterator(java.util.Iterator<org.apache.spark.sql.Row> rowIterator,
                                     org.apache.spark.sql.types.StructType rowType)

Method Detail

hasNext

public boolean hasNext()

next

public org.apache.spark.sql.Row next()