Package io.delta.tables

Class DeltaOptimizeBuilder

Object
io.delta.tables.DeltaOptimizeBuilder
All Implemented Interfaces:
org.apache.spark.sql.delta.util.AnalysisHelper
public class DeltaOptimizeBuilder extends Object implements org.apache.spark.sql.delta.util.AnalysisHelper
Builder class for constructing OPTIMIZE command and executing.

param: sparkSession SparkSession to use for execution param: tableIdentifier Id of the table on which to execute the optimize param: options Hadoop file system options for read and write.

Since:
2.0.0

  • Nested Class Summary

    Nested classes/interfaces inherited from interface org.apache.spark.sql.delta.util.AnalysisHelper

    org.apache.spark.sql.delta.util.AnalysisHelper.FakeLogicalPlan, org.apache.spark.sql.delta.util.AnalysisHelper.FakeLogicalPlan$

  • Constructor Summary

    Constructors
    Constructor
    Description
    DeltaOptimizeBuilder()
     

  • Method Summary

    Modifier and Type
    Method
    Description
    org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>
    executeCompaction()
    Compact the small files in selected partitions.
    org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>
    executeZOrderBy(String... columns)
    Z-Order the data in selected partitions using the given columns.
    org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>
    executeZOrderBy(scala.collection.immutable.Seq<String> columns)
    Z-Order the data in selected partitions using the given columns.
    DeltaOptimizeBuilder
    where(String partitionFilter)
    Apply partition filter on this optimize command builder to limit the operation on selected partitions.

    Methods inherited from class java.lang.Object

    equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    Methods inherited from interface org.apache.spark.sql.delta.util.AnalysisHelper

    improveUnsupportedOpError, resolveReferencesForExpressions, toDataset, tryResolveReferences, tryResolveReferencesForExpressions, tryResolveReferencesForExpressions

  • Constructor Details

    • DeltaOptimizeBuilder

      public DeltaOptimizeBuilder()

  • Method Details

    • executeZOrderBy

      public org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> executeZOrderBy(String... columns)
      Z-Order the data in selected partitions using the given columns.
      Parameters:
      columns - Zero or more columns to order the data using Z-Order curves
      Returns:
      DataFrame containing the OPTIMIZE execution metrics
      Since:
      2.0.0

    • where

      public DeltaOptimizeBuilder where(String partitionFilter)
      Apply partition filter on this optimize command builder to limit the operation on selected partitions.
      Parameters:
      partitionFilter - The partition filter to apply
      Returns:
      DeltaOptimizeBuilder with partition filter applied
      Since:
      2.0.0

    • executeCompaction

      public org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> executeCompaction()
      Compact the small files in selected partitions.
      Returns:
      DataFrame containing the OPTIMIZE execution metrics
      Since:
      2.0.0

    • executeZOrderBy

      public org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> executeZOrderBy(scala.collection.immutable.Seq<String> columns)
      Z-Order the data in selected partitions using the given columns.
      Parameters:
      columns - Zero or more columns to order the data using Z-Order curves
      Returns:
      DataFrame containing the OPTIMIZE execution metrics
      Since:
      2.0.0