A B C D E F G H I J K L M N O P R S T U V W

A

add(StructField) - Method in class io.delta.kernel.types.StructType

add(String, DataType) - Method in class io.delta.kernel.types.StructType

add(String, DataType, boolean) - Method in class io.delta.kernel.types.StructType

add(String, DataType, FieldMetadata) - Method in class io.delta.kernel.types.StructType

add(String, DataType, boolean, FieldMetadata) - Method in class io.delta.kernel.types.StructType

ALWAYS_FALSE - Static variable in class io.delta.kernel.expressions.AlwaysFalse

ALWAYS_TRUE - Static variable in class io.delta.kernel.expressions.AlwaysTrue

AlwaysFalse - Class in io.delta.kernel.expressions

Predicate which always evaluates to false.

AlwaysTrue - Class in io.delta.kernel.expressions

Predicate which always evaluates to true.

And - Class in io.delta.kernel.expressions

AND expression

And(Predicate, Predicate) - Constructor for class io.delta.kernel.expressions.And

ArrayType - Class in io.delta.kernel.types

Represent array data type

ArrayType(DataType, boolean) - Constructor for class io.delta.kernel.types.ArrayType

ArrayValue - Interface in io.delta.kernel.data

Abstraction to represent a single array value in a ColumnVector.

at(int) - Method in class io.delta.kernel.types.StructType

B

BasePrimitiveType - Class in io.delta.kernel.types

Base class for all primitive types DataType.

BINARY - Static variable in class io.delta.kernel.types.BinaryType

BinaryType - Class in io.delta.kernel.types

The data type representing byte[] values.

BOOLEAN - Static variable in class io.delta.kernel.types.BooleanType

BooleanType - Class in io.delta.kernel.types

Data type representing boolean type values.

build() - Method in interface io.delta.kernel.ScanBuilder

build(Engine) - Method in interface io.delta.kernel.TransactionBuilder

Build the transaction.

build() - Method in class io.delta.kernel.types.FieldMetadata.Builder

builder() - Static method in class io.delta.kernel.types.FieldMetadata

Builder() - Constructor for class io.delta.kernel.types.FieldMetadata.Builder

BYTE - Static variable in class io.delta.kernel.types.ByteType

ByteType - Class in io.delta.kernel.types

The data type representing byte type values.

C

checkpoint(Engine, long) - Method in interface io.delta.kernel.Table

Checkpoint the table at given version.

CheckpointAlreadyExistsException - Exception in io.delta.kernel.exceptions

Thrown when trying to create a checkpoint at version v, but there already exists a checkpoint at version v.

CheckpointAlreadyExistsException(long) - Constructor for exception io.delta.kernel.exceptions.CheckpointAlreadyExistsException

close() - Method in interface io.delta.kernel.data.ColumnVector

Cleans up memory for this column vector.

CloseableIterable<T> - Interface in io.delta.kernel.utils

Extend the Java Iterable interface to provide a way to close the iterator.

CloseableIterator<T> - Interface in io.delta.kernel.utils

Closeable extension of Iterator

Column - Class in io.delta.kernel.expressions

An expression type that refers to a column (case-sensitive) in the input.

Column(String) - Constructor for class io.delta.kernel.expressions.Column

Create a column expression for referring to a column.

Column(String[]) - Constructor for class io.delta.kernel.expressions.Column

Create a column expression to refer to a nested column.

column(int) - Method in class io.delta.kernel.types.StructType

Creates a Column expression for the field at the given ordinal

ColumnarBatch - Interface in io.delta.kernel.data

Represents zero or more rows of records with same schema type.

ColumnVector - Interface in io.delta.kernel.data

Represents zero or more values of a single column.

combine(CloseableIterator<T>) - Method in interface io.delta.kernel.utils.CloseableIterator

Combine the current iterator with another iterator.

commit(Engine, CloseableIterable<Row>) - Method in interface io.delta.kernel.Transaction

Commit the transaction including the data action rows generated by Transaction.generateAppendActions(io.delta.kernel.engine.Engine, io.delta.kernel.data.Row, io.delta.kernel.utils.CloseableIterator<io.delta.kernel.utils.DataFileStatus>, io.delta.kernel.DataWriteContext).

ConcurrentTransactionException - Exception in io.delta.kernel.exceptions

Thrown when concurrent transaction both attempt to update the table with same transaction identifier set through TransactionBuilder.withTransactionId(Engine, String, long) (String)}.

ConcurrentTransactionException(String, long, long) - Constructor for exception io.delta.kernel.exceptions.ConcurrentTransactionException

ConcurrentWriteException - Exception in io.delta.kernel.exceptions

Thrown when a concurrent transaction has written data after the current transaction has started.

ConcurrentWriteException() - Constructor for exception io.delta.kernel.exceptions.ConcurrentWriteException

ConcurrentWriteException(String) - Constructor for exception io.delta.kernel.exceptions.ConcurrentWriteException

contains(String) - Method in class io.delta.kernel.types.FieldMetadata

containsNull() - Method in class io.delta.kernel.types.ArrayType

create(Configuration) - Static method in class io.delta.kernel.defaults.engine.DefaultEngine

Create an instance of DefaultEngine.

createPrimitive(String) - Static method in class io.delta.kernel.types.BasePrimitiveType

Create a primitive type DataType

createSelectionVector(boolean[], int, int) - Method in class io.delta.kernel.defaults.engine.DefaultExpressionHandler

createSelectionVector(boolean[], int, int) - Method in interface io.delta.kernel.engine.ExpressionHandler

Create a selection vector, a boolean type ColumnVector, on top of the range of values given in values array.

createTransactionBuilder(Engine, String, Operation) - Method in interface io.delta.kernel.Table

Create a TransactionBuilder which can create a Transaction object to mutate the table.

D

DataFileStatistics - Class in io.delta.kernel.utils

Statistics about data file in a Delta Lake table.

DataFileStatistics(long, Map<Column, Literal>, Map<Column, Literal>, Map<Column, Long>) - Constructor for class io.delta.kernel.utils.DataFileStatistics

Create a new instance of DataFileStatistics.

DataFileStatus - Class in io.delta.kernel.utils

Extends FileStatus to include additional details such as column level statistics of the data file in the Delta Lake table.

DataFileStatus(String, long, long, Optional<DataFileStatistics>) - Constructor for class io.delta.kernel.utils.DataFileStatus

Create a new instance of DataFileStatus.

DataType - Class in io.delta.kernel.types

Base class for all data types.

DataType() - Constructor for class io.delta.kernel.types.DataType

DataWriteContext - Interface in io.delta.kernel

Contains the context for writing data to Delta table.

DATE - Static variable in class io.delta.kernel.types.DateType

DateType - Class in io.delta.kernel.types

A date type, supporting "0001-01-01" through "9999-12-31".

DecimalType - Class in io.delta.kernel.types

The data type representing java.math.BigDecimal values.

DecimalType(int, int) - Constructor for class io.delta.kernel.types.DecimalType

DefaultEngine - Class in io.delta.kernel.defaults.engine

Default implementation of Engine based on Hadoop APIs.

DefaultExpressionHandler - Class in io.delta.kernel.defaults.engine

Default implementation of ExpressionHandler

DefaultExpressionHandler() - Constructor for class io.delta.kernel.defaults.engine.DefaultExpressionHandler

DefaultFileSystemClient - Class in io.delta.kernel.defaults.engine

Default implementation of FileSystemClient based on Hadoop APIs.

DefaultFileSystemClient(Configuration) - Constructor for class io.delta.kernel.defaults.engine.DefaultFileSystemClient

Create an instance of the default FileSystemClient implementation.

DefaultJsonHandler - Class in io.delta.kernel.defaults.engine

Default implementation of JsonHandler based on Hadoop APIs.

DefaultJsonHandler(Configuration) - Constructor for class io.delta.kernel.defaults.engine.DefaultJsonHandler

DefaultParquetHandler - Class in io.delta.kernel.defaults.engine

Default implementation of ParquetHandler based on Hadoop APIs.

DefaultParquetHandler(Configuration) - Constructor for class io.delta.kernel.defaults.engine.DefaultParquetHandler

Create an instance of default ParquetHandler implementation.

deserializeStructType(String) - Method in class io.delta.kernel.defaults.engine.DefaultJsonHandler

deserializeStructType(String) - Method in interface io.delta.kernel.engine.JsonHandler

Deserialize the Delta schema from structTypeJson according to the Delta Protocol schema serialization rules .

DOUBLE - Static variable in class io.delta.kernel.types.DoubleType

DoubleType - Class in io.delta.kernel.types

The data type representing double type values.

E

empty() - Static method in class io.delta.kernel.types.FieldMetadata

emptyIterable() - Static method in interface io.delta.kernel.utils.CloseableIterable

Return an CloseableIterable object for an empty collection.

Engine - Interface in io.delta.kernel.engine

Interface encapsulating all clients needed by the Delta Kernel in order to read the Delta table.

equals(Object) - Method in class io.delta.kernel.expressions.Column

equals(Object) - Method in class io.delta.kernel.types.ArrayType

equals(Object) - Method in class io.delta.kernel.types.BasePrimitiveType

equals(Object) - Method in class io.delta.kernel.types.DataType

equals(Object) - Method in class io.delta.kernel.types.DecimalType

equals(Object) - Method in class io.delta.kernel.types.FieldMetadata

equals(Object) - Method in class io.delta.kernel.types.MapType

equals(Object) - Method in class io.delta.kernel.types.StructField

equals(Object) - Method in class io.delta.kernel.types.StructType

equivalent(DataType) - Method in class io.delta.kernel.types.ArrayType

equivalent(DataType) - Method in class io.delta.kernel.types.DataType

Are the data types same? The metadata or column names could be different.

equivalent(DataType) - Method in class io.delta.kernel.types.MapType

equivalent(DataType) - Method in class io.delta.kernel.types.StructType

eval(ColumnarBatch) - Method in interface io.delta.kernel.expressions.ExpressionEvaluator

Evaluate the expression on given ColumnarBatch data.

eval(ColumnarBatch, Optional<ColumnVector>) - Method in interface io.delta.kernel.expressions.PredicateEvaluator

Evaluate the predicate on given inputData.

Evolving - Annotation Type in io.delta.kernel.annotation

APIs that are meant to evolve towards becoming stable APIs, but are not stable APIs yet.

Expression - Interface in io.delta.kernel.expressions

Base interface for all Kernel expressions.

ExpressionEvaluator - Interface in io.delta.kernel.expressions

Interface for implementing an Expression evaluator.

ExpressionHandler - Interface in io.delta.kernel.engine

Provides expression evaluation capability to Delta Kernel.

F

FieldMetadata - Class in io.delta.kernel.types

The metadata for a given StructField.

FieldMetadata.Builder - Class in io.delta.kernel.types

Builder class for FieldMetadata.

fieldNames() - Method in class io.delta.kernel.types.StructType

fields() - Method in class io.delta.kernel.types.StructType

FileReadRequest - Interface in io.delta.kernel.engine

Represents a request to read a range of bytes from a given file.

FileStatus - Class in io.delta.kernel.utils

Class for encapsulating metadata about a file in Delta Lake table.

FileSystemClient - Interface in io.delta.kernel.engine

Provides file system related functionalities to Delta Kernel.

filter(Function<T, Boolean>) - Method in interface io.delta.kernel.utils.CloseableIterator

FilteredColumnarBatch - Class in io.delta.kernel.data

Represents a filtered version of ColumnarBatch.

FilteredColumnarBatch(ColumnarBatch, Optional<ColumnVector>) - Constructor for class io.delta.kernel.data.FilteredColumnarBatch

FLOAT - Static variable in class io.delta.kernel.types.FloatType

FloatType - Class in io.delta.kernel.types

The data type representing float type values.

forEach(Consumer<? super T>) - Method in interface io.delta.kernel.utils.CloseableIterable

forPath(Engine, String) - Static method in interface io.delta.kernel.Table

Instantiate a table object for the Delta Lake table at the given path.

G

generateAppendActions(Engine, Row, CloseableIterator<DataFileStatus>, DataWriteContext) - Static method in interface io.delta.kernel.Transaction

For given data files, generate Delta actions that can be committed in a transaction.

get(String) - Method in class io.delta.kernel.types.FieldMetadata

get(String) - Method in class io.delta.kernel.types.StructType

getAllPrimitiveTypes() - Static method in class io.delta.kernel.types.BasePrimitiveType

For testing only

getArray(int) - Method in interface io.delta.kernel.data.ColumnVector

Return the array value located at rowId.

getArray(int) - Method in interface io.delta.kernel.data.Row

Return array value of the column located at the given ordinal.

getBinary(int) - Method in interface io.delta.kernel.data.ColumnVector

Returns the binary type value for rowId.

getBinary(int) - Method in interface io.delta.kernel.data.Row

Return binary value of the column located at the given ordinal.

getBoolean(int) - Method in interface io.delta.kernel.data.ColumnVector

Returns the boolean type value for rowId.

getBoolean(int) - Method in interface io.delta.kernel.data.Row

Return boolean value of the column located at the given ordinal.

getByte(int) - Method in interface io.delta.kernel.data.ColumnVector

Returns the byte type value for rowId.

getByte(int) - Method in interface io.delta.kernel.data.Row

Return byte value of the column located at the given ordinal.

getChild(int) - Method in interface io.delta.kernel.data.ColumnVector

Get the child vector associated with the given ordinal.

getChildren() - Method in class io.delta.kernel.expressions.Column

getChildren() - Method in interface io.delta.kernel.expressions.Expression

getChildren() - Method in class io.delta.kernel.expressions.Literal

getChildren() - Method in class io.delta.kernel.expressions.PartitionValueExpression

getChildren() - Method in class io.delta.kernel.expressions.ScalarExpression

getColumnVector(int) - Method in interface io.delta.kernel.data.ColumnarBatch

Return the ColumnVector for the given ordinal in the columnar batch.

getData() - Method in class io.delta.kernel.data.FilteredColumnarBatch

Return the data as ColumnarBatch.

getDataType() - Method in interface io.delta.kernel.data.ColumnVector

getDataType() - Method in class io.delta.kernel.expressions.Literal

Get the datatype of the literal object.

getDataType() - Method in class io.delta.kernel.expressions.PartitionValueExpression

Get the data type of the partition value.

getDataType() - Method in class io.delta.kernel.types.StructField

getDecimal(int) - Method in interface io.delta.kernel.data.ColumnVector

Returns the decimal type value for rowId.

getDecimal(int) - Method in interface io.delta.kernel.data.Row

Return decimal value of the column located at the given ordinal.

getDescription() - Method in enum io.delta.kernel.Operation

Returns the string that will be recorded in the transaction log.

getDouble(int) - Method in interface io.delta.kernel.data.ColumnVector

Returns the double type value for rowId.

getDouble(int) - Method in interface io.delta.kernel.data.Row

Return double value of the column located at the given ordinal.

getElements() - Method in interface io.delta.kernel.data.ArrayValue

A ColumnVector containing the array elements with exactly ArrayValue.getSize() elements.

getElementType() - Method in class io.delta.kernel.types.ArrayType

getEntries() - Method in class io.delta.kernel.types.FieldMetadata

getEvaluator(StructType, Expression, DataType) - Method in class io.delta.kernel.defaults.engine.DefaultExpressionHandler

getEvaluator(StructType, Expression, DataType) - Method in interface io.delta.kernel.engine.ExpressionHandler

Create an ExpressionEvaluator that can evaluate the given expression on ColumnarBatchs with the given batchSchema.

getExpressionHandler() - Method in class io.delta.kernel.defaults.engine.DefaultEngine

getExpressionHandler() - Method in interface io.delta.kernel.engine.Engine

Get the connector provided ExpressionHandler.

getFileSystemClient() - Method in class io.delta.kernel.defaults.engine.DefaultEngine

getFileSystemClient() - Method in interface io.delta.kernel.engine.Engine

Get the connector provided FileSystemClient.

getFloat(int) - Method in interface io.delta.kernel.data.ColumnVector

Returns the float type value for rowId.

getFloat(int) - Method in interface io.delta.kernel.data.Row

Return float value of the column located at the given ordinal.

getInput() - Method in class io.delta.kernel.expressions.PartitionValueExpression

Get the expression reference to the serialized partition value.

getInt(int) - Method in interface io.delta.kernel.data.ColumnVector

Returns the int type value for rowId.

getInt(int) - Method in interface io.delta.kernel.data.Row

Return integer value of the column located at the given ordinal.

getJsonHandler() - Method in class io.delta.kernel.defaults.engine.DefaultEngine

getJsonHandler() - Method in interface io.delta.kernel.engine.Engine

Get the connector provided JsonHandler.

getKeys() - Method in interface io.delta.kernel.data.MapValue

A ColumnVector containing the keys.

getKeyType() - Method in class io.delta.kernel.types.MapType

getLatestSnapshot(Engine) - Method in interface io.delta.kernel.Table

Get the latest snapshot of the table.

getLeft() - Method in class io.delta.kernel.expressions.And

getLeft() - Method in class io.delta.kernel.expressions.Or

getLong(int) - Method in interface io.delta.kernel.data.ColumnVector

Returns the long type value for rowId.

getLong(int) - Method in interface io.delta.kernel.data.Row

Return long value of the column located at the given ordinal.

getMap(int) - Method in interface io.delta.kernel.data.ColumnVector

Return the map value located at rowId.

getMap(int) - Method in interface io.delta.kernel.data.Row

Return map value of the column located at the given ordinal.

getMaxValues() - Method in class io.delta.kernel.utils.DataFileStatistics

Get the maximum values of the columns in the data file.

getMessage() - Method in exception io.delta.kernel.exceptions.TableAlreadyExistsException

getMetadata() - Method in class io.delta.kernel.types.StructField

getMinValues() - Method in class io.delta.kernel.utils.DataFileStatistics

Get the minimum values of the columns in the data file.

getModificationTime() - Method in class io.delta.kernel.utils.FileStatus

Get the modification time of the file in epoch millis.

getName() - Method in class io.delta.kernel.expressions.ScalarExpression

getName() - Method in class io.delta.kernel.types.StructField

getNames() - Method in class io.delta.kernel.expressions.Column

getNullCounts() - Method in class io.delta.kernel.utils.DataFileStatistics

Get the number of nulls of columns in the data file.

getNumRecords() - Method in class io.delta.kernel.utils.DataFileStatistics

Get the number of records in the data file.

getParquetHandler() - Method in class io.delta.kernel.defaults.engine.DefaultEngine

getParquetHandler() - Method in interface io.delta.kernel.engine.Engine

Get the connector provided ParquetHandler.

getPartitionColumns(Engine) - Method in interface io.delta.kernel.Transaction

Get the list of logical names of the partition columns.

getPath() - Method in interface io.delta.kernel.engine.FileReadRequest

Get the fully qualified path of the file from which to read the data.

getPath(Engine) - Method in interface io.delta.kernel.Table

The fully qualified path of this Table instance.

getPath() - Method in class io.delta.kernel.utils.FileStatus

Get the path to the file.

getPrecision() - Method in class io.delta.kernel.types.DecimalType

getPredicateEvaluator(StructType, Predicate) - Method in class io.delta.kernel.defaults.engine.DefaultExpressionHandler

getPredicateEvaluator(StructType, Predicate) - Method in interface io.delta.kernel.engine.ExpressionHandler

Create a PredicateEvaluator that can evaluate the given predicate expression and return a selection vector (ColumnVector of boolean type).

getReadLength() - Method in interface io.delta.kernel.engine.FileReadRequest

Get the length of the data to read from the file starting at the startOffset.

getRemainingFilter() - Method in interface io.delta.kernel.Scan

Get the remaining filter that is not guaranteed to be satisfied for the data Delta Kernel returns.

getRight() - Method in class io.delta.kernel.expressions.And

getRight() - Method in class io.delta.kernel.expressions.Or

getRows() - Method in interface io.delta.kernel.data.ColumnarBatch

getRows() - Method in class io.delta.kernel.data.FilteredColumnarBatch

Iterator of rows that survived the filter.

getScale() - Method in class io.delta.kernel.types.DecimalType

getScanBuilder(Engine) - Method in interface io.delta.kernel.Snapshot

Create a scan builder to construct a Scan to read data from this snapshot.

getScanFiles(Engine) - Method in interface io.delta.kernel.Scan

Get an iterator of data files to scan.

getScanState(Engine) - Method in interface io.delta.kernel.Scan

Get the scan state associated with the current scan.

getSchema() - Method in interface io.delta.kernel.data.ColumnarBatch

getSchema() - Method in interface io.delta.kernel.data.Row

getSchema(Engine) - Method in interface io.delta.kernel.Snapshot

Get the schema of the table at this snapshot.

getSchema(Engine) - Method in interface io.delta.kernel.Transaction

Get the schema of the table.

getSelectionVector() - Method in class io.delta.kernel.data.FilteredColumnarBatch

Optional selection vector containing one entry for each row in data indicating whether a row is selected or not selected.

getShort(int) - Method in interface io.delta.kernel.data.ColumnVector

Returns the short type value for rowId.

getShort(int) - Method in interface io.delta.kernel.data.Row

Return short value of the column located at the given ordinal.

getSize() - Method in interface io.delta.kernel.data.ArrayValue

The number of elements in the array

getSize() - Method in interface io.delta.kernel.data.ColumnarBatch

getSize() - Method in interface io.delta.kernel.data.ColumnVector

getSize() - Method in interface io.delta.kernel.data.MapValue

The number of elements in the map

getSize() - Method in class io.delta.kernel.utils.FileStatus

Get the size of the file in bytes.

getSnapshotAsOfTimestamp(Engine, long) - Method in interface io.delta.kernel.Table

Get the snapshot of the table at the given timestamp.

getSnapshotAsOfVersion(Engine, long) - Method in interface io.delta.kernel.Table

Get the snapshot at the given versionId.

getStartOffset() - Method in interface io.delta.kernel.engine.FileReadRequest

Get the start offset in the file from where to start reading the data.

getStatistics() - Method in class io.delta.kernel.utils.DataFileStatus

Get the statistics of the data file encapsulated in this object.

getStatisticsColumns() - Method in interface io.delta.kernel.DataWriteContext

Returns the list of Column that the connector can optionally collect statistics.

getString(int) - Method in interface io.delta.kernel.data.ColumnVector

Returns the string type value for rowId.

getString(int) - Method in interface io.delta.kernel.data.Row

Return string value of the column located at the given ordinal.

getStruct(int) - Method in interface io.delta.kernel.data.Row

Return struct value of the column located at the given ordinal.

getTablePath() - Method in exception io.delta.kernel.exceptions.TableNotFoundException

getTargetDirectory() - Method in interface io.delta.kernel.DataWriteContext

Returns the target directory where the data should be written.

getTransactionState(Engine) - Method in interface io.delta.kernel.Transaction

Get the state of the transaction.

getValue() - Method in class io.delta.kernel.expressions.Literal

Get the literal value.

getValues() - Method in interface io.delta.kernel.data.MapValue

A ColumnVector containing the values.

getValueType() - Method in class io.delta.kernel.types.MapType

getVersion(Engine) - Method in interface io.delta.kernel.Snapshot

Get the version of this snapshot in the table.

getVersion() - Method in class io.delta.kernel.TransactionCommitResult

Contains the version of the transaction committed as.

getWriteContext(Engine, Row, Map<String, Literal>) - Static method in interface io.delta.kernel.Transaction

Get the context for writing data into a table.

H

hashCode() - Method in class io.delta.kernel.expressions.Column

hashCode() - Method in class io.delta.kernel.types.ArrayType

hashCode() - Method in class io.delta.kernel.types.BasePrimitiveType

hashCode() - Method in class io.delta.kernel.types.DataType

hashCode() - Method in class io.delta.kernel.types.DecimalType

hashCode() - Method in class io.delta.kernel.types.FieldMetadata

hashCode() - Method in class io.delta.kernel.types.MapType

hashCode() - Method in class io.delta.kernel.types.StructField

hashCode() - Method in class io.delta.kernel.types.StructType

hasNext() - Method in interface io.delta.kernel.utils.CloseableIterator

Returns true if the iteration has more elements.

I

indexOf(String) - Method in class io.delta.kernel.types.StructType

inMemoryIterable(CloseableIterator<T>) - Static method in interface io.delta.kernel.utils.CloseableIterable

Return an CloseableIterable object that is backed by an in-memory collection of given CloseableIterator.

INTEGER - Static variable in class io.delta.kernel.types.IntegerType

IntegerType - Class in io.delta.kernel.types

The data type representing integer type values.

io.delta.kernel - package io.delta.kernel

Delta Kernel interfaces for constructing table object representing a Delta Lake table, getting snapshot from the table and building a scan object to scan a subset of the data in the table.

io.delta.kernel.annotation - package io.delta.kernel.annotation

io.delta.kernel.data - package io.delta.kernel.data

Delta Kernel interfaces for representing data in columnar and row format.

io.delta.kernel.defaults.engine - package io.delta.kernel.defaults.engine

Default implementation of Engine interface and the sub-interfaces exposed by the Engine.

io.delta.kernel.engine - package io.delta.kernel.engine

Interfaces to allow the connector to bring their own implementation of functions such as reading parquet files, listing files in a file system, parsing a JSON string etc.

io.delta.kernel.exceptions - package io.delta.kernel.exceptions

io.delta.kernel.expressions - package io.delta.kernel.expressions

Expressions framework that defines the most common expressions which the connectors can use to pass predicates to Delta Kernel.

io.delta.kernel.types - package io.delta.kernel.types

Data types defined by the Delta Kernel to exchange the type info between the Delta Kernel and the connectors.

io.delta.kernel.utils - package io.delta.kernel.utils

Utilities.

isDataColumn() - Method in class io.delta.kernel.types.StructField

isMetadataColumn() - Method in class io.delta.kernel.types.StructField

isNullable() - Method in class io.delta.kernel.types.StructField

isNullAt(int) - Method in interface io.delta.kernel.data.ColumnVector

isNullAt(int) - Method in interface io.delta.kernel.data.Row

isPrimitiveType(String) - Static method in class io.delta.kernel.types.BasePrimitiveType

Is the given type name a primitive type?

isReadyForCheckpoint() - Method in class io.delta.kernel.TransactionCommitResult

Is the table ready for checkpoint (i.e.

isValueContainsNull() - Method in class io.delta.kernel.types.MapType

iterator() - Method in interface io.delta.kernel.utils.CloseableIterable

Overrides the default iterator method to return a CloseableIterator.

J

JsonHandler - Interface in io.delta.kernel.engine

Provides JSON handling functionality to Delta Kernel.

K

KernelEngineException - Exception in io.delta.kernel.exceptions

Throws when the Engine encountered an error while executing an operation.

KernelEngineException(String, Throwable) - Constructor for exception io.delta.kernel.exceptions.KernelEngineException

KernelException - Exception in io.delta.kernel.exceptions

Thrown when Kernel cannot execute the requested operation due to the operation being invalid or unsupported.

KernelException() - Constructor for exception io.delta.kernel.exceptions.KernelException

KernelException(String) - Constructor for exception io.delta.kernel.exceptions.KernelException

KernelException(Throwable) - Constructor for exception io.delta.kernel.exceptions.KernelException

KernelException(String, Throwable) - Constructor for exception io.delta.kernel.exceptions.KernelException

L

length() - Method in class io.delta.kernel.types.StructType

listFrom(String) - Method in class io.delta.kernel.defaults.engine.DefaultFileSystemClient

listFrom(String) - Method in interface io.delta.kernel.engine.FileSystemClient

List the paths in the same directory that are lexicographically greater or equal to (UTF-8 sorting) the given `path`.

Literal - Class in io.delta.kernel.expressions

A literal value.

LONG - Static variable in class io.delta.kernel.types.LongType

LongType - Class in io.delta.kernel.types

The data type representing long type values.

M

map(Function<T, U>) - Method in interface io.delta.kernel.utils.CloseableIterator

MapType - Class in io.delta.kernel.types

Data type representing a map type.

MapType(DataType, DataType, boolean) - Constructor for class io.delta.kernel.types.MapType

MapValue - Interface in io.delta.kernel.data

Abstraction to represent a single map value in a ColumnVector.

METADATA_ROW_INDEX_COLUMN - Static variable in class io.delta.kernel.types.StructField

METADATA_ROW_INDEX_COLUMN_NAME - Static variable in class io.delta.kernel.types.StructField

The name of a row index metadata column.

MetadataChangedException - Exception in io.delta.kernel.exceptions

Thrown when the metadata of the Delta table has changed between the time of transaction start and the time of commit.

MetadataChangedException() - Constructor for exception io.delta.kernel.exceptions.MetadataChangedException

mkdirs(String) - Method in class io.delta.kernel.defaults.engine.DefaultFileSystemClient

mkdirs(String) - Method in interface io.delta.kernel.engine.FileSystemClient

Create a directory at the given path including parent directories.

N

next() - Method in interface io.delta.kernel.utils.CloseableIterator

Returns the next element in the iteration.

O

of(String, long, long) - Static method in class io.delta.kernel.utils.FileStatus

Create a FileStatus with the given path, size and modification time.

ofBinary(byte[]) - Static method in class io.delta.kernel.expressions.Literal

Create a binary type literal expression.

ofBoolean(boolean) - Static method in class io.delta.kernel.expressions.Literal

Create a boolean type literal expression.

ofByte(byte) - Static method in class io.delta.kernel.expressions.Literal

Create a byte type literal expression.

ofDate(int) - Static method in class io.delta.kernel.expressions.Literal

Create a date type literal expression.

ofDecimal(BigDecimal, int, int) - Static method in class io.delta.kernel.expressions.Literal

Create a decimal type literal expression.

ofDouble(double) - Static method in class io.delta.kernel.expressions.Literal

Create a double type literal expression.

ofFloat(float) - Static method in class io.delta.kernel.expressions.Literal

Create a float type literal expression.

ofInt(int) - Static method in class io.delta.kernel.expressions.Literal

Create a integer type literal expression.

ofLong(long) - Static method in class io.delta.kernel.expressions.Literal

Create a long type literal expression.

ofNull(DataType) - Static method in class io.delta.kernel.expressions.Literal

Create null value literal.

ofShort(short) - Static method in class io.delta.kernel.expressions.Literal

Create a short type literal expression.

ofString(String) - Static method in class io.delta.kernel.expressions.Literal

Create a string type literal expression.

ofTimestamp(long) - Static method in class io.delta.kernel.expressions.Literal

Create a timestamp type literal expression.

ofTimestampNtz(long) - Static method in class io.delta.kernel.expressions.Literal

Create a timestamp_ntz type literal expression.

Operation - Enum in io.delta.kernel

An operation that can be performed on a Delta table.

Or - Class in io.delta.kernel.expressions

OR expression

Or(Predicate, Predicate) - Constructor for class io.delta.kernel.expressions.Or

P

ParquetHandler - Interface in io.delta.kernel.engine

Provides Parquet file related functionalities to Delta Kernel.

parseJson(ColumnVector, StructType, Optional<ColumnVector>) - Method in class io.delta.kernel.defaults.engine.DefaultJsonHandler

parseJson(ColumnVector, StructType, Optional<ColumnVector>) - Method in interface io.delta.kernel.engine.JsonHandler

Parse the given json strings and return the fields requested by outputSchema as columns in a ColumnarBatch.

PartitionValueExpression - Class in io.delta.kernel.expressions

Expression to decode the serialized partition value into partition type value according the Delta Protocol spec.

PartitionValueExpression(Expression, DataType) - Constructor for class io.delta.kernel.expressions.PartitionValueExpression

Create partition_value expression.

Predicate - Class in io.delta.kernel.expressions

Defines predicate scalar expression which is an extension of ScalarExpression that evaluates to true, false, or null for each input row.

Predicate(String, List<Expression>) - Constructor for class io.delta.kernel.expressions.Predicate

Predicate(String, Expression) - Constructor for class io.delta.kernel.expressions.Predicate

Constructor for a unary Predicate expression

Predicate(String, Expression, Expression) - Constructor for class io.delta.kernel.expressions.Predicate

Constructor for a binary Predicate expression

PredicateEvaluator - Interface in io.delta.kernel.expressions

Special interface for evaluating Predicate on input batch and return a selection vector containing one value for each row in input batch indicating whether the row has passed the predicate or not.

ProtocolChangedException - Exception in io.delta.kernel.exceptions

Thrown when the protocol of the Delta table has changed between the time of transaction start and the time of commit.

ProtocolChangedException(long) - Constructor for exception io.delta.kernel.exceptions.ProtocolChangedException

putBoolean(String, boolean) - Method in class io.delta.kernel.types.FieldMetadata.Builder

putBooleanArray(String, Boolean[]) - Method in class io.delta.kernel.types.FieldMetadata.Builder

putDouble(String, double) - Method in class io.delta.kernel.types.FieldMetadata.Builder

putDoubleArray(String, Double[]) - Method in class io.delta.kernel.types.FieldMetadata.Builder

putFieldMetadata(String, FieldMetadata) - Method in class io.delta.kernel.types.FieldMetadata.Builder

putFieldMetadataArray(String, FieldMetadata[]) - Method in class io.delta.kernel.types.FieldMetadata.Builder

putLong(String, long) - Method in class io.delta.kernel.types.FieldMetadata.Builder

putLongArray(String, Long[]) - Method in class io.delta.kernel.types.FieldMetadata.Builder

putNull(String) - Method in class io.delta.kernel.types.FieldMetadata.Builder

putString(String, String) - Method in class io.delta.kernel.types.FieldMetadata.Builder

putStringArray(String, String[]) - Method in class io.delta.kernel.types.FieldMetadata.Builder

R

readFiles(CloseableIterator<FileReadRequest>) - Method in class io.delta.kernel.defaults.engine.DefaultFileSystemClient

readFiles(CloseableIterator<FileReadRequest>) - Method in interface io.delta.kernel.engine.FileSystemClient

Return an iterator of byte streams one for each read request in readRequests.

readJsonFiles(CloseableIterator<FileStatus>, StructType, Optional<Predicate>) - Method in class io.delta.kernel.defaults.engine.DefaultJsonHandler

readJsonFiles(CloseableIterator<FileStatus>, StructType, Optional<Predicate>) - Method in interface io.delta.kernel.engine.JsonHandler

Read and parse the JSON format file at given locations and return the data as a ColumnarBatch with the columns requested by physicalSchema.

readParquetFiles(CloseableIterator<FileStatus>, StructType, Optional<Predicate>) - Method in class io.delta.kernel.defaults.engine.DefaultParquetHandler

readParquetFiles(CloseableIterator<FileStatus>, StructType, Optional<Predicate>) - Method in interface io.delta.kernel.engine.ParquetHandler

Read the Parquet format files at the given locations and return the data as a ColumnarBatch with the columns requested by physicalSchema.

resolvePath(String) - Method in class io.delta.kernel.defaults.engine.DefaultFileSystemClient

resolvePath(String) - Method in interface io.delta.kernel.engine.FileSystemClient

Resolve the given path to a fully qualified path.

Row - Interface in io.delta.kernel.data

Represent a single record

S

ScalarExpression - Class in io.delta.kernel.expressions

Scalar SQL expressions which take zero or more inputs and for each input row generate one output value.

ScalarExpression(String, List<Expression>) - Constructor for class io.delta.kernel.expressions.ScalarExpression

Scan - Interface in io.delta.kernel

Represents a scan of a Delta table.

ScanBuilder - Interface in io.delta.kernel

Builder to construct Scan object.

serializeAsJson() - Method in class io.delta.kernel.utils.DataFileStatistics

SHORT - Static variable in class io.delta.kernel.types.ShortType

ShortType - Class in io.delta.kernel.types

The data type representing short type values.

slice(int, int) - Method in interface io.delta.kernel.data.ColumnarBatch

Return a slice of the current batch.

Snapshot - Interface in io.delta.kernel

Represents the snapshot of a Delta table.

spliterator() - Method in interface io.delta.kernel.utils.CloseableIterable

STRING - Static variable in class io.delta.kernel.types.StringType

StringType - Class in io.delta.kernel.types

The data type representing string type values.

StructField - Class in io.delta.kernel.types

Represents a subfield of StructType with additional properties and metadata.

StructField(String, DataType, boolean) - Constructor for class io.delta.kernel.types.StructField

StructField(String, DataType, boolean, FieldMetadata) - Constructor for class io.delta.kernel.types.StructField

StructType - Class in io.delta.kernel.types

Struct type which contains one or more columns.

StructType() - Constructor for class io.delta.kernel.types.StructType

StructType(List<StructField>) - Constructor for class io.delta.kernel.types.StructType

T

Table - Interface in io.delta.kernel

Represents the Delta Lake table for a given path.

TableAlreadyExistsException - Exception in io.delta.kernel.exceptions

Thrown when trying to create a Delta table at a location where a Delta table already exists.

TableAlreadyExistsException(String, String) - Constructor for exception io.delta.kernel.exceptions.TableAlreadyExistsException

TableAlreadyExistsException(String) - Constructor for exception io.delta.kernel.exceptions.TableAlreadyExistsException

TableNotFoundException - Exception in io.delta.kernel.exceptions

Thrown when there is no Delta table at the given location.

TableNotFoundException(String) - Constructor for exception io.delta.kernel.exceptions.TableNotFoundException

TableNotFoundException(String, String) - Constructor for exception io.delta.kernel.exceptions.TableNotFoundException

TIMESTAMP - Static variable in class io.delta.kernel.types.TimestampType

TIMESTAMP_NTZ - Static variable in class io.delta.kernel.types.TimestampNTZType

TimestampNTZType - Class in io.delta.kernel.types

The timestamp without time zone type represents a local time in microsecond precision, which is independent of time zone.

TimestampType - Class in io.delta.kernel.types

A timestamp type, supporting [0001-01-01T00:00:00.000000Z, 9999-12-31T23:59:59.999999Z] where the left/right-bound is a date and time of the proleptic Gregorian calendar in UTC+00:00.

toJson() - Method in class io.delta.kernel.types.ArrayType

toJson() - Method in class io.delta.kernel.types.BasePrimitiveType

toJson() - Method in class io.delta.kernel.types.DataType

Convert the data type to Delta protocol specified serialization format.

toJson() - Method in class io.delta.kernel.types.DecimalType

toJson() - Method in class io.delta.kernel.types.FieldMetadata

toJson() - Method in class io.delta.kernel.types.MapType

toJson() - Method in class io.delta.kernel.types.StructField

toJson() - Method in class io.delta.kernel.types.StructType

toString() - Method in class io.delta.kernel.expressions.Column

toString() - Method in class io.delta.kernel.expressions.Literal

toString() - Method in class io.delta.kernel.expressions.PartitionValueExpression

toString() - Method in class io.delta.kernel.expressions.Predicate

toString() - Method in class io.delta.kernel.expressions.ScalarExpression

toString() - Method in class io.delta.kernel.types.ArrayType

toString() - Method in class io.delta.kernel.types.BasePrimitiveType

toString() - Method in class io.delta.kernel.types.DataType

toString() - Method in class io.delta.kernel.types.DecimalType

toString() - Method in class io.delta.kernel.types.FieldMetadata

toString() - Method in class io.delta.kernel.types.MapType

toString() - Method in class io.delta.kernel.types.StructField

toString() - Method in class io.delta.kernel.types.StructType

Transaction - Interface in io.delta.kernel

Represents a transaction to mutate a Delta table.

TransactionBuilder - Interface in io.delta.kernel

Builder for creating a Transaction to mutate a Delta table.

TransactionCommitResult - Class in io.delta.kernel

Contains the result of a successful transaction commit.

TransactionCommitResult(long, boolean) - Constructor for class io.delta.kernel.TransactionCommitResult

transformLogicalData(Engine, Row, CloseableIterator<FilteredColumnarBatch>, Map<String, Literal>) - Static method in interface io.delta.kernel.Transaction

Given the logical data that needs to be written to the table, convert it into the required physical data depending upon the table Delta protocol and features enabled on the table.

transformPhysicalData(Engine, Row, Row, CloseableIterator<ColumnarBatch>) - Static method in interface io.delta.kernel.Scan

Transform the physical data read from the table data file into the logical data that expected out of the Delta table.

U

USER_DEFAULT - Static variable in class io.delta.kernel.types.DecimalType

V

valueOf(String) - Static method in enum io.delta.kernel.Operation

Returns the enum constant of this type with the specified name.

values() - Static method in enum io.delta.kernel.Operation

Returns an array containing the constants of this enum type, in the order they are declared.

W

withDeletedColumnAt(int) - Method in interface io.delta.kernel.data.ColumnarBatch

Return a copy of this ColumnarBatch with the column at given ordinal removed.

withFilter(Engine, Predicate) - Method in interface io.delta.kernel.ScanBuilder

Apply the given filter expression to prune any files that do not contain data satisfying the given filter.

withNewColumn(int, StructField, ColumnVector) - Method in interface io.delta.kernel.data.ColumnarBatch

Return a copy of the ColumnarBatch with given new column vector inserted at the given columnVector at given ordinal.

withNewSchema(StructType) - Method in interface io.delta.kernel.data.ColumnarBatch

Generate a copy of this ColumnarBatch with the given newSchema.

withPartitionColumns(Engine, List<String>) - Method in interface io.delta.kernel.TransactionBuilder

Set the list of partitions columns when create a new partitioned table.

withReadSchema(Engine, StructType) - Method in interface io.delta.kernel.ScanBuilder

Apply the given readSchema.

withSchema(Engine, StructType) - Method in interface io.delta.kernel.TransactionBuilder

Set the schema of the table when creating a new table.

withTransactionId(Engine, String, long) - Method in interface io.delta.kernel.TransactionBuilder

Set the transaction identifier for idempotent writes.

writeJsonFileAtomically(String, CloseableIterator<Row>, boolean) - Method in class io.delta.kernel.defaults.engine.DefaultJsonHandler

Makes use of LogStore implementations in `delta-storage` to atomically write the data to a file depending upon the destination filesystem.

writeJsonFileAtomically(String, CloseableIterator<Row>, boolean) - Method in interface io.delta.kernel.engine.JsonHandler

Serialize each Row in the iterator as JSON and write as a separate line in destination file.

writeParquetFileAtomically(String, CloseableIterator<FilteredColumnarBatch>) - Method in class io.delta.kernel.defaults.engine.DefaultParquetHandler

Makes use of LogStore implementations in `delta-storage` to atomically write the data to a file depending upon the destination filesystem.

writeParquetFileAtomically(String, CloseableIterator<FilteredColumnarBatch>) - Method in interface io.delta.kernel.engine.ParquetHandler

Write the given data as a Parquet file.

writeParquetFiles(String, CloseableIterator<FilteredColumnarBatch>, List<Column>) - Method in class io.delta.kernel.defaults.engine.DefaultParquetHandler

writeParquetFiles(String, CloseableIterator<FilteredColumnarBatch>, List<Column>) - Method in interface io.delta.kernel.engine.ParquetHandler

Write the given data batches to a Parquet files.

A B C D E F G H I J K L M N O P R S T U V W