io.delta.kernel.engine

Interface CommitCoordinatorClientHandler

All Known Implementing Classes:

DefaultCommitCoordinatorClientHandler

@Evolving
public interface CommitCoordinatorClientHandler

An interface that encapsulates all the functions needed by Kernel to perform commits to a table owned by a Commit Coordinator. Commit coordinator is defined by the Delta Protocol.

Since:

3.3.0

See Also:

Coordinated commit protocol table feature.

Method Summary

Modifier and Type	Method and Description
void	backfillToVersion(String logPath, java.util.Map<String,String> tableConf, long version, Long lastKnownBackfilledVersion) Requests the commit coordinator client to backfill commits up to a specified version.
CommitResponse	commit(String logPath, java.util.Map<String,String> tableConf, long commitVersion, CloseableIterator<Row> actions, UpdatedActions updatedActions) Commits a set of actions to a specified table at a given version.
GetCommitsResponse	getCommits(String logPath, java.util.Map<String,String> tableConf, Long startVersion, Long endVersion) Retrieves unbackfilled commits for a specified table within a given version range.
java.util.Map<String,String>	registerTable(String logPath, long currentVersion, AbstractMetadata currentMetadata, AbstractProtocol currentProtocol) API to register the table represented by the given `logPath` at the provided currentTableVersion with the commit coordinator this commit coordinator client represents.
boolean	semanticEquals(CommitCoordinatorClientHandler other) Compares this CommitCoordinatorClientHandler for semantic equality with another instance.

Method Detail

registerTable

java.util.Map<String,String> registerTable(String logPath,
                                           long currentVersion,
                                           AbstractMetadata currentMetadata,
                                           AbstractProtocol currentProtocol)

API to register the table represented by the given `logPath` at the provided currentTableVersion with the commit coordinator this commit coordinator client represents.

This API is called when the table is being converted from a file system table to a coordinated-commit table.

When a new coordinated-commit table is being created, the currentTableVersion will be -1 and the upgrade commit needs to be a file system commit which will write the backfilled file directly.

Parameters:

logPath - The path to the delta log of the table that should be converted

currentVersion - The currentTableVersion is the version of the table just before conversion. currentTableVersion + 1 represents the commit that will do the conversion. This must be backfilled atomically. currentTableVersion + 2 represents the first commit after conversion. This will go through the CommitCoordinatorClient and the client is free to choose when it wants to backfill this commit.

currentMetadata - The metadata of the table at currentTableVersion

currentProtocol - The protocol of the table at currentTableVersion

Returns:

A map of key-value pairs which is issued by the commit coordinator to identify the table. This should be stored in the table's metadata. This information needs to be passed to the commit(java.lang.String, java.util.Map<java.lang.String, java.lang.String>, long, io.delta.kernel.utils.CloseableIterator<io.delta.kernel.data.Row>, io.delta.kernel.engine.coordinatedcommits.UpdatedActions), getCommits(java.lang.String, java.util.Map<java.lang.String, java.lang.String>, java.lang.Long, java.lang.Long), and backfillToVersion(java.lang.String, java.util.Map<java.lang.String, java.lang.String>, long, java.lang.Long) APIs to identify the table.

commit

CommitResponse commit(String logPath,
                      java.util.Map<String,String> tableConf,
                      long commitVersion,
                      CloseableIterator<Row> actions,
                      UpdatedActions updatedActions)
               throws CommitFailedException

Commits a set of actions to a specified table at a given version.

This method applies the provided actions to the table identified by logPath as given table version commitVersion.

Parameters:

logPath - The path to the delta log of the target table

tableConf - Configuration details returned by the commit coordinator client during table registration

commitVersion - The version of the commit that is being committed.

actions - The set of actions to be committed

updatedActions - Additional information for the commit, including:

• Commit info
• Metadata changes
• Protocol changes

Returns:

CommitResponse containing the file status of the committed file. Note: If the commit is already backfilled, the file status may be omitted, and the client can retrieve this information independently.

Throws:

CommitFailedException - if the commit operation fails

getCommits

GetCommitsResponse getCommits(String logPath,
                              java.util.Map<String,String> tableConf,
                              Long startVersion,
                              Long endVersion)

Retrieves unbackfilled commits for a specified table within a given version range.

This method fetches commits that have not yet been backfilled for the table identified by logPath. It returns commits between startVersion and endVersion (inclusive), ignoring commits outside this range. The returned commits are guaranteed to be contiguous and in ascending version order.

Note: The first returned version may not equal startVersion if some versions have already been backfilled and are no longer tracked by the commit coordinator.

The latestTableVersion in the response represents the highest commit version ratified by the commit coordinator. A value of -1 is only valid if the commit coordinator has never ratified any version (i.e., never accepted any unbackfilled commit).

Parameters:

logPath - The path to the delta log of the target table

tableConf - The table configuration returned by the commit coordinator during registration

startVersion - The minimum commit version to retrieve (inclusive, can be null)

endVersion - The maximum commit version to retrieve (inclusive, can be null)

Returns:

GetCommitsResponse containing:

• A list of Commit objects
• The latestTableVersion tracked by CommitCoordinatorClientHandler

backfillToVersion

void backfillToVersion(String logPath,
                       java.util.Map<String,String> tableConf,
                       long version,
                       Long lastKnownBackfilledVersion)
                throws java.io.IOException

Requests the commit coordinator client to backfill commits up to a specified version.

This method instructs the commit coordinator client to backfill all commits up to the given version and notify the commit coordinator of the completion. A successful return from this method guarantees that the backfill has been completed, even if the commit coordinator hasn't been notified yet.

Parameters:

logPath - The path to the delta log of the table to be backfilled

tableConf - The table configuration returned by the commit coordinator during registration

version - The target version up to which commits should be backfilled

lastKnownBackfilledVersion - The most recent version known to be backfilled before this call. If null or invalid, the client will backfill from the table's beginning. Can be null.

Throws:

java.io.IOException - if the backfill operation fails

Note: A successful return indicates backfill completion, but the commit coordinator may not be immediately aware of this update.

semanticEquals

boolean semanticEquals(CommitCoordinatorClientHandler other)

Compares this CommitCoordinatorClientHandler for semantic equality with another instance.

Semantic equality is defined by the specific implementation of CommitCoordinatorClientHandler. Two instances are considered semantically equal if they can be used interchangeably for any CommitCoordinatorClientHandler API, including but not limited to:

• commit(java.lang.String, java.util.Map<java.lang.String, java.lang.String>, long, io.delta.kernel.utils.CloseableIterator<io.delta.kernel.data.Row>, io.delta.kernel.engine.coordinatedcommits.UpdatedActions)
• getCommits(java.lang.String, java.util.Map<java.lang.String, java.lang.String>, java.lang.Long, java.lang.Long)

For example, semantic equality might be based on both instances pointing to the same underlying endpoint.

Parameters:

other - The CommitCoordinatorClientHandler to compare with this instance

Returns:

boolean True if the instances are semantically equal, false otherwise