io.delta.tables

Class DeltaOptimizeBuilder

Object

io.delta.tables.DeltaOptimizeBuilder

All Implemented Interfaces:

org.apache.spark.sql.delta.util.AnalysisHelper

public class DeltaOptimizeBuilder
extends Object
implements org.apache.spark.sql.delta.util.AnalysisHelper

Builder class for constructing OPTIMIZE command and executing.

param: sparkSession SparkSession to use for execution param: tableIdentifier Id of the table on which to execute the optimize param: options Hadoop file system options for read and write.

Since:

2.0.0

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.spark.sql.delta.util.AnalysisHelper

org.apache.spark.sql.delta.util.AnalysisHelper.FakeLogicalPlan, org.apache.spark.sql.delta.util.AnalysisHelper.FakeLogicalPlan$

Constructor Summary

Constructors

Constructor and Description
DeltaOptimizeBuilder()

Method Summary

Modifier and Type	Method and Description
org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>	executeCompaction() Compact the small files in selected partitions.
org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>	executeZOrderBy(scala.collection.Seq<String> columns) Z-Order the data in selected partitions using the given columns.
org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>	executeZOrderBy(String... columns) Z-Order the data in selected partitions using the given columns.
DeltaOptimizeBuilder	where(String partitionFilter) Apply partition filter on this optimize command builder to limit the operation on selected partitions.

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.sql.delta.util.AnalysisHelper

$init$, improveUnsupportedOpError, resolveReferencesForExpressions, toDataset, tryResolveReferences, tryResolveReferencesForExpressions, tryResolveReferencesForExpressions

Constructor Detail

DeltaOptimizeBuilder

public DeltaOptimizeBuilder()

Method Detail

executeZOrderBy

public org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> executeZOrderBy(String... columns)

Z-Order the data in selected partitions using the given columns.

Parameters:

columns - Zero or more columns to order the data using Z-Order curves

Returns:

DataFrame containing the OPTIMIZE execution metrics

Since:

2.0.0

where

public DeltaOptimizeBuilder where(String partitionFilter)

Apply partition filter on this optimize command builder to limit the operation on selected partitions.

Parameters:

partitionFilter - The partition filter to apply

Returns:

DeltaOptimizeBuilder with partition filter applied

Since:

2.0.0

executeCompaction

public org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> executeCompaction()

Compact the small files in selected partitions.

Returns:

DataFrame containing the OPTIMIZE execution metrics

Since:

2.0.0

executeZOrderBy

public org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> executeZOrderBy(scala.collection.Seq<String> columns)

Z-Order the data in selected partitions using the given columns.

Parameters:

columns - Zero or more columns to order the data using Z-Order curves

Returns:

DataFrame containing the OPTIMIZE execution metrics

Since:

2.0.0