Class DeltaOptimizeBuilder

  • All Implemented Interfaces:
    org.apache.spark.sql.delta.util.AnalysisHelper

    public class DeltaOptimizeBuilder
    extends Object
    implements org.apache.spark.sql.delta.util.AnalysisHelper
    Builder class for constructing OPTIMIZE command and executing.

    param: sparkSession SparkSession to use for execution param: tableIdentifier Id of the table on which to execute the optimize

    Since:
    2.0.0
    • Nested Class Summary

      • Nested classes/interfaces inherited from interface org.apache.spark.sql.delta.util.AnalysisHelper

        org.apache.spark.sql.delta.util.AnalysisHelper.FakeLogicalPlan, org.apache.spark.sql.delta.util.AnalysisHelper.FakeLogicalPlan$
    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> executeCompaction()
      Compact the small files in selected partitions.
      org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> executeZOrderBy​(scala.collection.Seq<String> columns)
      Z-Order the data in selected partitions using the given columns.
      DeltaOptimizeBuilder where​(String partitionFilter)
      Apply partition filter on this optimize command builder to limit the operation on selected partitions.
      • Methods inherited from class java.lang.Object

        equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
      • Methods inherited from interface org.apache.spark.sql.delta.util.AnalysisHelper

        improveUnsupportedOpError, resolveReferencesForExpressions, toDataset, tryResolveReferences, tryResolveReferencesForExpressions
    • Constructor Detail

      • DeltaOptimizeBuilder

        public DeltaOptimizeBuilder()
    • Method Detail

      • where

        public DeltaOptimizeBuilder where​(String partitionFilter)
        Apply partition filter on this optimize command builder to limit the operation on selected partitions.
        Parameters:
        partitionFilter - The partition filter to apply
        Since:
        2.0.0
      • executeCompaction

        public org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> executeCompaction()
        Compact the small files in selected partitions.
        Since:
        2.0.0
      • executeZOrderBy

        public org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> executeZOrderBy​(scala.collection.Seq<String> columns)
        Z-Order the data in selected partitions using the given columns.
        Parameters:
        columns - Zero or more columns to order the data using Z-Order curves
        Since:
        2.0.0