org.apache.iceberg

Class BaseFileScanTask

java.lang.Object

org.apache.iceberg.BaseFileScanTask

All Implemented Interfaces:

java.io.Serializable, ContentScanTask<DataFile>, FileScanTask, PartitionScanTask, ScanTask, SplittableScanTask<FileScanTask>

public class BaseFileScanTask
extends java.lang.Object
implements FileScanTask

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
BaseFileScanTask(DataFile file, DeleteFile[] deletes, java.lang.String schemaString, java.lang.String specString, ResidualEvaluator residuals)

Method Summary

Modifier and Type	Method and Description
java.util.List<DeleteFile>	deletes() A list of delete files to apply when reading the task's data file.
long	estimatedRowsCount() The estimated number of rows produced by this scan task.
F	file() The file to scan.
int	filesCount() The number of files that will be opened by this scan task.
long	length() The number of bytes to scan from the ContentScanTask.start() position in the file.
protected FileScanTask	newSplitTask(FileScanTask parentTask, long offset, long length)
Expression	residual() Returns the residual expression that should be applied to rows in this file scan.
Schema	schema() Return the schema for this file scan task.
protected FileScanTask	self()
long	sizeBytes() The number of bytes that should be read by this scan task.
PartitionSpec	spec() Returns the spec of the partition for this scan task
java.lang.Iterable<ThisT>	split(long targetSplitSize) Attempts to split this scan task into several smaller scan tasks, each close to splitSize size.
long	start() The starting position of this scan range in the file.
java.lang.String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.iceberg.FileScanTask

asFileScanTask, isFileScanTask

Methods inherited from interface org.apache.iceberg.ContentScanTask

estimatedRowsCount, file, length, partition, residual, start

Methods inherited from interface org.apache.iceberg.PartitionScanTask

spec

Methods inherited from interface org.apache.iceberg.ScanTask

asCombinedScanTask, asDataTask, isDataTask

Methods inherited from interface org.apache.iceberg.SplittableScanTask

split

Constructor Detail

BaseFileScanTask

public BaseFileScanTask(DataFile file,
                        DeleteFile[] deletes,
                        java.lang.String schemaString,
                        java.lang.String specString,
                        ResidualEvaluator residuals)

Method Detail

self

protected FileScanTask self()

newSplitTask

protected FileScanTask newSplitTask(FileScanTask parentTask,
                                    long offset,
                                    long length)

deletes

public java.util.List<DeleteFile> deletes()

Description copied from interface: FileScanTask

A list of delete files to apply when reading the task's data file.

Specified by:

deletes in interface FileScanTask

Returns:

a list of delete files to apply

sizeBytes

public long sizeBytes()

Description copied from interface: ScanTask

The number of bytes that should be read by this scan task.

Specified by:

sizeBytes in interface ContentScanTask<DataFile>

Specified by:

sizeBytes in interface FileScanTask

Specified by:

sizeBytes in interface ScanTask

Returns:

the total number of bytes to read

filesCount

public int filesCount()

Description copied from interface: ScanTask

The number of files that will be opened by this scan task.

Specified by:

filesCount in interface FileScanTask

Specified by:

filesCount in interface ScanTask

Returns:

the number of files to open

schema

public Schema schema()

Description copied from interface: FileScanTask

Return the schema for this file scan task.

Specified by:

schema in interface FileScanTask

file

public F file()

Description copied from interface: ContentScanTask

The file to scan.

Specified by:

file in interface ContentScanTask<F extends ContentFile<F>>

Returns:

the file to scan

spec

public PartitionSpec spec()

Description copied from interface: PartitionScanTask

Returns the spec of the partition for this scan task

Specified by:

spec in interface PartitionScanTask

start

public long start()

Description copied from interface: ContentScanTask

The starting position of this scan range in the file.

Specified by:

start in interface ContentScanTask<F extends ContentFile<F>>

Returns:

the start position of this scan range

length

public long length()

Description copied from interface: ContentScanTask

The number of bytes to scan from the ContentScanTask.start() position in the file.

Specified by:

length in interface ContentScanTask<F extends ContentFile<F>>

Returns:

the length of this scan range in bytes

residual

public Expression residual()

Description copied from interface: ContentScanTask

Returns the residual expression that should be applied to rows in this file scan.

The residual expression for a file is a filter expression created by partially evaluating the scan's filter using the file's partition data.

Specified by:

residual in interface ContentScanTask<F extends ContentFile<F>>

Returns:

a residual expression to apply to rows from this scan

estimatedRowsCount

public long estimatedRowsCount()

Description copied from interface: ScanTask

The estimated number of rows produced by this scan task.

Specified by:

estimatedRowsCount in interface ContentScanTask<F extends ContentFile<F>>

Specified by:

estimatedRowsCount in interface ScanTask

Returns:

the estimated number of produced rows

split

public java.lang.Iterable<ThisT> split(long targetSplitSize)

Description copied from interface: SplittableScanTask

Attempts to split this scan task into several smaller scan tasks, each close to splitSize size.

Note the target split size is just guidance and the actual split size may be either smaller or larger. File formats like Parquet may leverage the row group offset information while splitting tasks.

Specified by:

split in interface SplittableScanTask<ThisT extends ContentScanTask<F>>

Parameters:

targetSplitSize - the target size of each new scan task in bytes

Returns:

an Iterable of smaller tasks

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object