Package org.apache.iceberg.spark.data

Class SparkAvroWriter

java.lang.Object

org.apache.iceberg.spark.data.SparkAvroWriter

All Implemented Interfaces:

org.apache.avro.io.DatumWriter<org.apache.spark.sql.catalyst.InternalRow>, MetricsAwareDatumWriter<org.apache.spark.sql.catalyst.InternalRow>

public class SparkAvroWriter
extends Object
implements MetricsAwareDatumWriter<org.apache.spark.sql.catalyst.InternalRow>

Constructor Summary

Constructors

Constructor	Description
SparkAvroWriter(org.apache.spark.sql.types.StructType dsSchema)

Method Summary

Modifier and Type	Method	Description
Stream<FieldMetrics>	metrics()	Returns a stream of FieldMetrics that this MetricsAwareDatumWriter keeps track of.
void	setSchema(org.apache.avro.Schema schema)
void	write(org.apache.spark.sql.catalyst.InternalRow datum, org.apache.avro.io.Encoder out)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

SparkAvroWriter

public SparkAvroWriter(org.apache.spark.sql.types.StructType dsSchema)

Method Details

setSchema

public void setSchema(org.apache.avro.Schema schema)

Specified by:

setSchema in interface org.apache.avro.io.DatumWriter<org.apache.spark.sql.catalyst.InternalRow>

write

public void write(org.apache.spark.sql.catalyst.InternalRow datum,
 org.apache.avro.io.Encoder out)
           throws IOException

Specified by:

write in interface org.apache.avro.io.DatumWriter<org.apache.spark.sql.catalyst.InternalRow>

Throws:

IOException

metrics

public Stream<FieldMetrics> metrics()

Description copied from interface: MetricsAwareDatumWriter

Returns a stream of FieldMetrics that this MetricsAwareDatumWriter keeps track of.

Specified by:

metrics in interface MetricsAwareDatumWriter<org.apache.spark.sql.catalyst.InternalRow>