squirrel.driver.file

Module Contents

Classes

FileDriver

Drives the access to a data source.
class squirrel.driver.file.FileDriver(url: squirrel.constants.URL, storage_options: dict[str, Any] | None = None, **kwargs)

Bases: squirrel.driver.driver.Driver

Drives the access to a data source.

Initializes FileDriver.

Parameters

  • url (URL) – URL to file. Prefix with a protocol like s3:// or gs:// to read from other filesystems. For a full list of supported types, refer to fsspec.open().

  • storage_options (dict[str, Any] | None) – A dict with keyword arguments passed to file system initializer. Example of storage_options if you want to enable fsspec caching: storage_options={“protocol”: “simplecache”, “target_protocol”: “gs”, “cache_storage”: “path/to/cache”}

  • **kwargs – Keyword arguments passed to the super class initializer.

name = file
open(mode: str = 'r', create_if_not_exists: bool = False, **kwargs) → IO

Returns a handler for the file.

Uses squirrel.fsspec.fs.get_fs_from_url() to get a filesystem object corresponding to self.url. Simply returns the handler returned from the open() method of the filesystem.

Parameters

  • mode (str) – IO mode to use when opening the file. Will be forwarded to filesystem.open() method. Defaults to “r”.

  • create_if_not_exists (bool) – If True, the file will be created if it does not exist (along with the parent directories). This is achieved by providing auto_mkdir=create_if_not_exists as a storage option to the filesystem. No matter what you set in the FileDriver’s storage_options, create_if_not_exists will override the key auto_mkdir. Defaults to False.

  • **kwargs – Keyword arguments that are passed to the filesystem.open() method.

Returns

(IO) File handler for the file at self.path.