Class DeltaTableBuilder
Builder to specify how to create / replace a Delta table. You must specify the table name or the path before executing the builder. You can specify the table columns, the partitioning columns, the location of the data, the table comment and the property, and how you want to create / replace the Delta table.
After executing the builder, an instance of DeltaTable
is returned.
Scala example to create a Delta table with generated columns, using the table name:
val table: DeltaTable = DeltaTable.create()
.tableName("testTable")
.addColumn("c1", dataType = "INT", nullable = false)
.addColumn(
DeltaTable.columnBuilder("c2")
.dataType("INT")
.generatedAlwaysAs("c1 + 10")
.build()
)
.addColumn(
DeltaTable.columnBuilder("c3")
.dataType("INT")
.comment("comment")
.nullable(true)
.build()
)
.partitionedBy("c1", "c2")
.execute()
Scala example to create a delta table using the location:
val table: DeltaTable = DeltaTable.createIfNotExists(spark)
.location("/foo/`bar`")
.addColumn("c1", dataType = "INT", nullable = false)
.addColumn(
DeltaTable.columnBuilder(spark, "c2")
.dataType("INT")
.generatedAlwaysAs("c1 + 10")
.build()
)
.addColumn(
DeltaTable.columnBuilder(spark, "c3")
.dataType("INT")
.comment("comment")
.nullable(true)
.build()
)
.partitionedBy("c1", "c2")
.execute()
Java Example to replace a table:
DeltaTable table = DeltaTable.replace()
.tableName("db.table")
.addColumn("c1", "INT", false)
.addColumn(
DeltaTable.columnBuilder("c2")
.dataType("INT")
.generatedAlwaysBy("c1 + 10")
.build()
)
.execute();
- Since:
- 1.0.0
-
Method Summary
Modifier and TypeMethodDescription:: Evolving :::: Evolving :::: Evolving :::: Evolving ::addColumn
(org.apache.spark.sql.types.StructField col) :: Evolving ::addColumns
(org.apache.spark.sql.types.StructType cols) :: Evolving :::: Evolving :::: Evolving :::: Evolving ::execute()
:: Evolving :::: Evolving ::partitionedBy
(String... colNames) :: Evolving ::partitionedBy
(scala.collection.immutable.Seq<String> colNames) :: Evolving :::: Evolving :::: Evolving ::
-
Method Details
-
addColumn
:: Evolving ::Specify a column.
- Parameters:
colName
- string the column namedataType
- string the DDL data type- Returns:
- (undocumented)
- Since:
- 1.0.0
-
addColumn
:: Evolving ::Specify a column.
- Parameters:
colName
- string the column namedataType
- dataType the DDL data type- Returns:
- (undocumented)
- Since:
- 1.0.0
-
addColumn
:: Evolving ::Specify a column.
- Parameters:
colName
- string the column namedataType
- string the DDL data typenullable
- boolean whether the column is nullable- Returns:
- (undocumented)
- Since:
- 1.0.0
-
addColumn
public DeltaTableBuilder addColumn(String colName, org.apache.spark.sql.types.DataType dataType, boolean nullable) :: Evolving ::Specify a column.
- Parameters:
colName
- string the column namedataType
- dataType the DDL data typenullable
- boolean whether the column is nullable- Returns:
- (undocumented)
- Since:
- 1.0.0
-
addColumn
:: Evolving ::Specify a column.
- Parameters:
col
- structField the column struct- Returns:
- (undocumented)
- Since:
- 1.0.0
-
addColumns
:: Evolving ::Specify columns with an existing schema.
- Parameters:
cols
- structType the existing schema for columns- Returns:
- (undocumented)
- Since:
- 1.0.0
-
clusterBy
:: Evolving ::Specify the columns to cluster the output on the file system.
Note: This should only include table columns already defined in schema.
- Parameters:
colNames
- string* column names for clustering- Returns:
- (undocumented)
- Since:
- 3.2.0
-
clusterBy
:: Evolving ::Specify the columns to cluster the output on the file system.
Note: This should only include table columns already defined in schema.
- Parameters:
colNames
- string* column names for clustering- Returns:
- (undocumented)
- Since:
- 3.2.0
-
comment
:: Evolving ::Specify the table comment to describe the table.
- Parameters:
comment
- string table comment- Returns:
- (undocumented)
- Since:
- 1.0.0
-
execute
:: Evolving ::Execute the command to create / replace a Delta table and returns a instance of
DeltaTable
.- Returns:
- (undocumented)
- Since:
- 1.0.0
-
location
:: Evolving ::Specify the path to the directory where table data is stored, which could be a path on distributed storage.
- Parameters:
location
- string the data location- Returns:
- (undocumented)
- Since:
- 1.0.0
-
partitionedBy
:: Evolving ::Specify the columns to partition the output on the file system.
Note: This should only include table columns already defined in schema.
- Parameters:
colNames
- string* column names for partitioning- Returns:
- (undocumented)
- Since:
- 1.0.0
-
partitionedBy
:: Evolving ::Specify the columns to partition the output on the file system.
Note: This should only include table columns already defined in schema.
- Parameters:
colNames
- string* column names for partitioning- Returns:
- (undocumented)
- Since:
- 1.0.0
-
property
:: Evolving ::Specify a key-value pair to tag the table definition.
- Parameters:
key
- string the table property keyvalue
- string the table property value- Returns:
- (undocumented)
- Since:
- 1.0.0
-
tableName
:: Evolving ::Specify the table name, optionally qualified with a database name [database_name.] table_name
- Parameters:
identifier
- string the table name- Returns:
- (undocumented)
- Since:
- 1.0.0
-