io.delta.kernel.client

Interface FileHandler

All Known Subinterfaces:

JsonHandler, ParquetHandler

All Known Implementing Classes:

DefaultFileHandler, DefaultJsonHandler, DefaultParquetHandler

@Evolving
public interface FileHandler

Provides file handling functionality to Delta Kernel. Connectors can implement this client to provide Delta Kernel their own custom implementation of file splitting, additional predicate pushdown or any other connector-specific capabilities.

Since:

3.0.0

Method Summary

Modifier and Type	Method and Description
CloseableIterator<FileReadContext>	contextualizeFileReads(CloseableIterator<Row> fileIter, Predicate predicate) Associates a connector specific FileReadContext for each scan file represented by a Row in fileIter.

Method Detail

contextualizeFileReads

CloseableIterator<FileReadContext> contextualizeFileReads(CloseableIterator<Row> fileIter,
                                                          Predicate predicate)

Associates a connector specific FileReadContext for each scan file represented by a Row in fileIter. Delta Kernel will supply the returned FileReadContexts back to the connector when reading the file (for example, in ParquetHandler.readParquetFiles(io.delta.kernel.utils.CloseableIterator<io.delta.kernel.client.FileReadContext>, io.delta.kernel.types.StructType)). Delta Kernel does not interpret FileReadContext.

For example, a connector can attach split information in its own implementation of FileReadContext or attach any predicates.

Parameters:

fileIter - iterator of scan files where each Row contains FileStatus information

predicate - Predicate to prune data. This is optional for the connector to use for further optimization. Filtering by this predicate is not required.