io.delta.tables

Class DeltaTableBuilder

Object

io.delta.tables.DeltaTableBuilder

public class DeltaTableBuilder
extends Object

:: Evolving ::

Builder to specify how to create / replace a Delta table. You must specify the table name or the path before executing the builder. You can specify the table columns, the partitioning columns, the location of the data, the table comment and the property, and how you want to create / replace the Delta table.

After executing the builder, an instance of DeltaTable is returned.

Scala example to create a Delta table with generated columns, using the table name:

   val table: DeltaTable = DeltaTable.create()
     .tableName("testTable")
     .addColumn("c1",  dataType = "INT", nullable = false)
     .addColumn(
       DeltaTable.columnBuilder("c2")
         .dataType("INT")
         .generatedAlwaysAs("c1 + 10")
         .build()
     )
     .addColumn(
       DeltaTable.columnBuilder("c3")
         .dataType("INT")
         .comment("comment")
         .nullable(true)
         .build()
     )
     .partitionedBy("c1", "c2")
     .execute()
 

Scala example to create a delta table using the location:

   val table: DeltaTable = DeltaTable.createIfNotExists(spark)
     .location("/foo/`bar`")
     .addColumn("c1", dataType = "INT", nullable = false)
     .addColumn(
       DeltaTable.columnBuilder(spark, "c2")
         .dataType("INT")
         .generatedAlwaysAs("c1 + 10")
         .build()
     )
     .addColumn(
       DeltaTable.columnBuilder(spark, "c3")
         .dataType("INT")
         .comment("comment")
         .nullable(true)
         .build()
     )
     .partitionedBy("c1", "c2")
     .execute()
 

Java Example to replace a table:

   DeltaTable table = DeltaTable.replace()
     .tableName("db.table")
     .addColumn("c1",  "INT", false)
     .addColumn(
       DeltaTable.columnBuilder("c2")
         .dataType("INT")
         .generatedAlwaysBy("c1 + 10")
         .build()
     )
     .execute();
 

Since:

1.0.0

Method Summary

Modifier and Type	Method and Description
DeltaTableBuilder	addColumn(String colName, org.apache.spark.sql.types.DataType dataType) :: Evolving ::
DeltaTableBuilder	addColumn(String colName, org.apache.spark.sql.types.DataType dataType, boolean nullable) :: Evolving ::
DeltaTableBuilder	addColumn(String colName, String dataType) :: Evolving ::
DeltaTableBuilder	addColumn(String colName, String dataType, boolean nullable) :: Evolving ::
DeltaTableBuilder	addColumn(org.apache.spark.sql.types.StructField col) :: Evolving ::
DeltaTableBuilder	addColumns(org.apache.spark.sql.types.StructType cols) :: Evolving ::
DeltaTableBuilder	clusterBy(scala.collection.Seq<String> colNames) :: Evolving ::
DeltaTableBuilder	clusterBy(String... colNames) :: Evolving ::
DeltaTableBuilder	comment(String comment) :: Evolving ::
DeltaTable	execute() :: Evolving ::
DeltaTableBuilder	location(String location) :: Evolving ::
DeltaTableBuilder	partitionedBy(scala.collection.Seq<String> colNames) :: Evolving ::
DeltaTableBuilder	partitionedBy(String... colNames) :: Evolving ::
DeltaTableBuilder	property(String key, String value) :: Evolving ::
DeltaTableBuilder	tableName(String identifier) :: Evolving ::

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

addColumn

public DeltaTableBuilder addColumn(String colName,
                                   String dataType)

:: Evolving ::

Specify a column.

Parameters:

colName - string the column name

dataType - string the DDL data type

Returns:

(undocumented)

Since:

1.0.0

addColumn

public DeltaTableBuilder addColumn(String colName,
                                   org.apache.spark.sql.types.DataType dataType)

:: Evolving ::

Specify a column.

Parameters:

colName - string the column name

dataType - dataType the DDL data type

Returns:

(undocumented)

Since:

1.0.0

addColumn

public DeltaTableBuilder addColumn(String colName,
                                   String dataType,
                                   boolean nullable)

:: Evolving ::

Specify a column.

Parameters:

colName - string the column name

dataType - string the DDL data type

nullable - boolean whether the column is nullable

Returns:

(undocumented)

Since:

1.0.0

addColumn

public DeltaTableBuilder addColumn(String colName,
                                   org.apache.spark.sql.types.DataType dataType,
                                   boolean nullable)

:: Evolving ::

Specify a column.

Parameters:

colName - string the column name

dataType - dataType the DDL data type

nullable - boolean whether the column is nullable

Returns:

(undocumented)

Since:

1.0.0

addColumn

public DeltaTableBuilder addColumn(org.apache.spark.sql.types.StructField col)

:: Evolving ::

Specify a column.

Parameters:

col - structField the column struct

Returns:

(undocumented)

Since:

1.0.0

addColumns

public DeltaTableBuilder addColumns(org.apache.spark.sql.types.StructType cols)

:: Evolving ::

Specify columns with an existing schema.

Parameters:

cols - structType the existing schema for columns

Returns:

(undocumented)

Since:

1.0.0

clusterBy

public DeltaTableBuilder clusterBy(String... colNames)

:: Evolving ::

Specify the columns to cluster the output on the file system.

Note: This should only include table columns already defined in schema.

Parameters:

colNames - string* column names for clustering

Returns:

(undocumented)

Since:

3.2.0

clusterBy

public DeltaTableBuilder clusterBy(scala.collection.Seq<String> colNames)

:: Evolving ::

Specify the columns to cluster the output on the file system.

Note: This should only include table columns already defined in schema.

Parameters:

colNames - string* column names for clustering

Returns:

(undocumented)

Since:

3.2.0

comment

public DeltaTableBuilder comment(String comment)

:: Evolving ::

Specify the table comment to describe the table.

Parameters:

comment - string table comment

Returns:

(undocumented)

Since:

1.0.0

execute

public DeltaTable execute()

:: Evolving ::

Execute the command to create / replace a Delta table and returns a instance of DeltaTable.

Returns:

(undocumented)

Since:

1.0.0

location

public DeltaTableBuilder location(String location)

:: Evolving ::

Specify the path to the directory where table data is stored, which could be a path on distributed storage.

Parameters:

location - string the data location

Returns:

(undocumented)

Since:

1.0.0

partitionedBy

public DeltaTableBuilder partitionedBy(String... colNames)

:: Evolving ::

Specify the columns to partition the output on the file system.

Note: This should only include table columns already defined in schema.

Parameters:

colNames - string* column names for partitioning

Returns:

(undocumented)

Since:

1.0.0

partitionedBy

public DeltaTableBuilder partitionedBy(scala.collection.Seq<String> colNames)

:: Evolving ::

Specify the columns to partition the output on the file system.

Note: This should only include table columns already defined in schema.

Parameters:

colNames - string* column names for partitioning

Returns:

(undocumented)

Since:

1.0.0

property

public DeltaTableBuilder property(String key,
                                  String value)

:: Evolving ::

Specify a key-value pair to tag the table definition.

Parameters:

key - string the table property key

value - string the table property value

Returns:

(undocumented)

Since:

1.0.0

tableName

public DeltaTableBuilder tableName(String identifier)

:: Evolving ::

Specify the table name, optionally qualified with a database name [database_name.] table_name

Parameters:

identifier - string the table name

Returns:

(undocumented)

Since:

1.0.0