io.delta.flink.source

Class RowDataBoundedDeltaSourceBuilder

Object

<any>

io.delta.flink.source.RowDataBoundedDeltaSourceBuilder

public class RowDataBoundedDeltaSourceBuilder
extends <any>

A builder class for DeltaSource for a stream of RowData where the created source instance will operate in Bounded mode.

For most common use cases use DeltaSource.forBoundedRowData(org.apache.flink.core.fs.Path, org.apache.hadoop.conf.Configuration) utility method to instantiate the source. After instantiation of this builder you can either call build() method to get the instance of a DeltaSource or configure additional options using builder's API.

Method Summary

Modifier and Type	Method and Description
DeltaSource<org.apache.flink.table.data.RowData>	build() Creates an instance of DeltaSource for a stream of RowData.
RowDataBoundedDeltaSourceBuilder	columnNames(java.util.List<String> columnNames) Specifies a List of column names that should be read from Delta table.
RowDataBoundedDeltaSourceBuilder	columnNames(String... columnNames) Specifies an array of column names that should be read from Delta table.
RowDataBoundedDeltaSourceBuilder	option(String optionName, boolean optionValue) Sets a configuration option.
RowDataBoundedDeltaSourceBuilder	option(String optionName, int optionValue) Sets a configuration option.
RowDataBoundedDeltaSourceBuilder	option(String optionName, long optionValue) Sets a configuration option.
RowDataBoundedDeltaSourceBuilder	option(String optionName, String optionValue) Sets a configuration option.
RowDataBoundedDeltaSourceBuilder	timestampAsOf(String snapshotTimestamp) Sets value of "timestampAsOf" option.
RowDataBoundedDeltaSourceBuilder	versionAsOf(long snapshotVersion) Sets value of "versionAsOf" option.

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

columnNames

public RowDataBoundedDeltaSourceBuilder columnNames(java.util.List<String> columnNames)

Specifies a List of column names that should be read from Delta table. If this method is not used, Source will read all columns from Delta table.

If provided List is null or contains null, empty or blank elements it will throw a DeltaSourceValidationException by builder after calling build() method.

Parameters:

columnNames - column names that should be read.

columnNames

public RowDataBoundedDeltaSourceBuilder columnNames(String... columnNames)

Specifies an array of column names that should be read from Delta table. If this method is not used, Source will read all columns from Delta table.

If provided List is null or contains null, empty or blank elements it will throw a DeltaSourceValidationException by builder after calling build() method.

Parameters:

columnNames - column names that should be read.

versionAsOf

public RowDataBoundedDeltaSourceBuilder versionAsOf(long snapshotVersion)

Sets value of "versionAsOf" option. With this option we will load the given table version and read from it.

This option is mutually exclusive with timestampAsOf(String) option.

Parameters:

snapshotVersion - Delta table version to time travel to.

timestampAsOf

public RowDataBoundedDeltaSourceBuilder timestampAsOf(String snapshotTimestamp)

Sets value of "timestampAsOf" option. With this option we will load the latest table version that was generated at or before the given timestamp.

This option is mutually exclusive with versionAsOf(long) option.

Parameters:

snapshotTimestamp - The timestamp we should time travel to. Supported formats are:

• 2022-02-24
• 2022-02-24 04:55:00
• 2022-02-24 04:55:00.001
• 2022-02-24T04:55:00
• 2022-02-24T04:55:00.001
• 2022-02-24T04:55:00.001Z

option

public RowDataBoundedDeltaSourceBuilder option(String optionName,
                                               String optionValue)

Sets a configuration option.

Parameters:

optionName - Option name to set.

optionValue - Option String value to set.

option

public RowDataBoundedDeltaSourceBuilder option(String optionName,
                                               boolean optionValue)

Sets a configuration option.

Parameters:

optionName - Option name to set.

optionValue - Option boolean value to set.

option

public RowDataBoundedDeltaSourceBuilder option(String optionName,
                                               int optionValue)

Sets a configuration option.

Parameters:

optionName - Option name to set.

optionValue - Option int value to set.

option

public RowDataBoundedDeltaSourceBuilder option(String optionName,
                                               long optionValue)

Sets a configuration option.

Parameters:

optionName - Option name to set.

optionValue - Option long value to set.

build

public DeltaSource<org.apache.flink.table.data.RowData> build()

Creates an instance of DeltaSource for a stream of RowData. Created source will work in Bounded mode, meaning it will read the content of the configured Delta snapshot at the fixed version, ignoring all changes done to this table after starting this source.

This method can throw DeltaSourceValidationException in case of invalid arguments passed to Delta source builder.

Returns:

New DeltaSource instance.