Package io.delta.kernel.defaults.engine.fileio

Interface FileIO

All Known Implementing Classes:
HadoopFileIO
public interface FileIO
Interface for file IO operations. Connectors can implement their own version of the FileIO depending upon their environment. The DefaultEngine takes FileIO instance as input and all I/O operations from the default engine are done using the passed in FileIO instance.

  • Method Details

    • listFrom

      CloseableIterator<FileStatus> listFrom(String filePath) throws IOException
      List the paths in the same directory that are lexicographically greater or equal to (UTF-8 sorting) the given `path`. The result should also be sorted by the file name.
      Parameters:
      filePath - Fully qualified path to a file
      Returns:
      Closeable iterator of files. It is the responsibility of the caller to close the iterator.
      Throws:
      FileNotFoundException - if the file at the given path is not found
      IOException - for any other IO error.

    • getFileStatus

      FileStatus getFileStatus(String path) throws IOException
      Get the metadata of the file at the given path.
      Parameters:
      path - Fully qualified path to the file.
      Returns:
      Metadata of the file.
      Throws:
      IOException - for any IO error.

    • resolvePath

      String resolvePath(String path) throws IOException
      Resolve the given path to a fully qualified path.
      Parameters:
      path - Input path
      Returns:
      Fully qualified path.
      Throws:
      FileNotFoundException - If the given path doesn't exist.
      IOException - for any other IO error.

    • mkdirs

      boolean mkdirs(String path) throws IOException
      Create a directory at the given path including parent directories. This mimics the behavior of `mkdir -p` in Unix.
      Parameters:
      path - Full qualified path to create a directory at.
      Returns:
      true if the directory was created successfully, false otherwise.
      Throws:
      IOException - for any IO error.

    • newInputFile

      InputFile newInputFile(String path, long fileSize)
      Get an InputFile for file at given path which can be used to read the file from any arbitrary position in the file.
      Parameters:
      path - Fully qualified path to the file.
      fileSize - Size of the file in bytes.
      Returns:
      InputFile instance.

    • newOutputFile

      OutputFile newOutputFile(String path)
      Create a OutputFile to write new file at the given path.
      Parameters:
      path - Fully qualified path to the file.
      Returns:
      OutputFile instance which can be used to write to the file.

    • delete

      boolean delete(String path) throws IOException
      Delete the file at given path.
      Parameters:
      path - the path to delete. If path is a directory throws an exception.
      Returns:
      true if delete is successful else false.
      Throws:
      IOException - for any IO error.

    • getConf

      Optional<String> getConf(String confKey)
      Get the configuration value for the given key.

      TODO: should be in a separate interface? may be called ConfigurationProvider?

      Parameters:
      confKey - configuration key name
      Returns:
      If no such value is present, an Optional.empty() is returned.