public interface OverwriteFiles extends SnapshotUpdate<OverwriteFiles>
This API accumulates file additions and produces a new Snapshot of the table by replacing
all the files that match the filter expression with the set of additions. This operation is used
to implement idempotent writes that always replace a section of a table with new data.
Overwrites can be validated
When committing, these changes will be applied to the latest table snapshot. Commit conflicts will be resolved by applying the changes to the new latest snapshot and reattempting the commit. This has no requirements for the latest snapshot and will not fail based on other snapshot changes.
| Modifier and Type | Method and Description |
|---|---|
OverwriteFiles |
addFile(DataFile file)
Add a
DataFile to the table. |
OverwriteFiles |
overwriteByRowFilter(Expression expr)
Delete files that match an
Expression on data rows from the table. |
OverwriteFiles |
validateAddedFiles()
Signal that each file added to the table must match the overwrite expression.
|
deleteWith, setapply, commitOverwriteFiles overwriteByRowFilter(Expression expr)
Expression on data rows from the table.
A file is selected to be deleted by the expression if it could contain any rows that match the
expression (candidate files are selected using an
inclusive projection). These candidate files are
deleted if all of the rows in the file must match the expression (the partition data matches
the expression's Projections.strict(PartitionSpec) strict projection}). This guarantees
that files are deleted if and only if all rows in the file must match the expression.
Files that may contain some rows that match the expression and some rows that do not will
result in a ValidationException.
expr - an expression on rows in the tableValidationException - If a file can contain both rows that match and rows that do notOverwriteFiles addFile(DataFile file)
DataFile to the table.file - a data fileOverwriteFiles validateAddedFiles()
If this method is called, each added file is validated on commit to ensure that it matches the overwrite row filter. This is used to ensure that writes are idempotent: that files cannot be added during a commit that would not be removed if the operation were run a second time.