io.delta.kernel

Interface Scan

@Evolving
public interface Scan

Represents a scan of a Delta table.

Since:

3.0.0

Method Summary

Modifier and Type	Method and Description
java.util.Optional<Predicate>	getRemainingFilter() Get the remaining filter that is not guaranteed to be satisfied for the data Delta Kernel returns.
CloseableIterator<FilteredColumnarBatch>	getScanFiles(TableClient tableClient) Get an iterator of data files to scan.
Row	getScanState(TableClient tableClient) Get the scan state associated with the current scan.
static CloseableIterator<FilteredColumnarBatch>	readData(TableClient tableClient, Row scanState, CloseableIterator<Row> scanFileRowIter, java.util.Optional<Predicate> predicate) Get the data from the given scan files using the connector provided TableClient.

Method Detail

getScanFiles

CloseableIterator<FilteredColumnarBatch> getScanFiles(TableClient tableClient)

Get an iterator of data files to scan.

Parameters:

tableClient - TableClient instance to use in Delta Kernel.

getRemainingFilter

java.util.Optional<Predicate> getRemainingFilter()

Get the remaining filter that is not guaranteed to be satisfied for the data Delta Kernel returns. This filter is used by Delta Kernel to do data skipping when possible.

getScanState

Row getScanState(TableClient tableClient)

Get the scan state associated with the current scan. This state is common across all files in the scan to be read.

Parameters:

tableClient - TableClient instance to use in Delta Kernel.

readData

static CloseableIterator<FilteredColumnarBatch> readData(TableClient tableClient,
                                                         Row scanState,
                                                         CloseableIterator<Row> scanFileRowIter,
                                                         java.util.Optional<Predicate> predicate)
                                                  throws java.io.IOException

Get the data from the given scan files using the connector provided TableClient.

Parameters:

tableClient - Connector provided TableClient implementation.

scanState - Scan state returned by getScanState(TableClient)

scanFileRowIter - an iterator of Rows. Each Row represents one scan file from the FilteredColumnarBatch returned by getScanFiles(TableClient)

predicate - An optional predicate that can be used for data skipping while reading the scan files.

Throws:

java.io.IOException - when error occurs while reading the data.