Package org.apache.spark.sql.connector.iceberg.distributions

Class Distributions

  • public class Distributions
extends java.lang.Object
    Helper methods to create distributions to pass into Spark.
    Since:
    3.2.0

    • Method Summary

      All Methods Static Methods Concrete Methods 
      Modifier and Type Method Description
      static ClusteredDistribution clustered​(org.apache.spark.sql.connector.expressions.Expression[] clustering)
      Creates a distribution where tuples that share the same values for clustering expressions are co-located in the same partition.
      static OrderedDistribution ordered​(SortOrder[] ordering)
      Creates a distribution where tuples have been ordered across partitions according to ordering expressions, but not necessarily within a given partition.
      static UnspecifiedDistribution unspecified()
      Creates a distribution where no promises are made about co-location of data.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Method Detail

      • unspecified

        public static UnspecifiedDistribution unspecified()
        Creates a distribution where no promises are made about co-location of data.

      • clustered

        public static ClusteredDistribution clustered​(org.apache.spark.sql.connector.expressions.Expression[] clustering)
        Creates a distribution where tuples that share the same values for clustering expressions are co-located in the same partition.

      • ordered

        public static OrderedDistribution ordered​(SortOrder[] ordering)
        Creates a distribution where tuples have been ordered across partitions according to ordering expressions, but not necessarily within a given partition.