Package org.apache.iceberg.mr.mapred

Class MapredIcebergInputFormat<T>

java.lang.Object

org.apache.iceberg.mr.mapred.MapredIcebergInputFormat<T>

Type Parameters:

T - Java class of records constructed by Iceberg; default is Record

All Implemented Interfaces:

org.apache.hadoop.mapred.InputFormat<Void,Container<T>>

Direct Known Subclasses:

HiveIcebergInputFormat

public class MapredIcebergInputFormat<T>
extends Object
implements org.apache.hadoop.mapred.InputFormat<Void,Container<T>>

Generic MR v1 InputFormat API for Iceberg.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	MapredIcebergInputFormat.CompatibilityTaskAttemptContextImpl

Constructor Summary

Constructors

Constructor	Description
MapredIcebergInputFormat()

Method Summary

Modifier and Type	Method	Description
static InputFormatConfig.ConfigBuilder	configure(org.apache.hadoop.mapred.JobConf job)	Configures the JobConf to use the MapredIcebergInputFormat and returns a helper to add further configuration.
org.apache.hadoop.mapred.RecordReader<Void,Container<T>>	getRecordReader(org.apache.hadoop.mapred.InputSplit split, org.apache.hadoop.mapred.JobConf job, org.apache.hadoop.mapred.Reporter reporter)
org.apache.hadoop.mapred.InputSplit[]	getSplits(org.apache.hadoop.mapred.JobConf job, int numSplits)
static org.apache.hadoop.mapreduce.TaskAttemptContext	newTaskAttemptContext(org.apache.hadoop.mapred.JobConf job, org.apache.hadoop.mapred.Reporter reporter)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

MapredIcebergInputFormat

public MapredIcebergInputFormat()

Method Details

configure

public static InputFormatConfig.ConfigBuilder configure(org.apache.hadoop.mapred.JobConf job)

Configures the JobConf to use the MapredIcebergInputFormat and returns a helper to add further configuration.

Parameters:

job - the JobConf to configure

getSplits

public org.apache.hadoop.mapred.InputSplit[] getSplits(org.apache.hadoop.mapred.JobConf job,
 int numSplits)
                                                throws IOException

Specified by:

getSplits in interface org.apache.hadoop.mapred.InputFormat<Void,Container<T>>

Throws:

IOException

getRecordReader

public org.apache.hadoop.mapred.RecordReader<Void,Container<T>> getRecordReader(org.apache.hadoop.mapred.InputSplit split,
 org.apache.hadoop.mapred.JobConf job,
 org.apache.hadoop.mapred.Reporter reporter)
                                                                         throws IOException

Specified by:

getRecordReader in interface org.apache.hadoop.mapred.InputFormat<Void,Container<T>>

Throws:

IOException

newTaskAttemptContext

public static org.apache.hadoop.mapreduce.TaskAttemptContext newTaskAttemptContext(org.apache.hadoop.mapred.JobConf job,
 org.apache.hadoop.mapred.Reporter reporter)