squirrel.fsspec.fs

Module Contents

Functions

create_dir_if_does_not_exist(fs: fsspec.spec.AbstractFileSystem, path: str) → None	If the given path does not exist, creates a directory at it.
get_fs_from_url(url: squirrel.constants.URL, **storage_options) → squirrel.constants.FILESYSTEM	Get filesystem suitable for a url.
get_protocol(url: str) → str	Get the protocol from a url, return empty string if local

squirrel.fsspec.fs.create_dir_if_does_not_exist(fs: fsspec.spec.AbstractFileSystem, path: str) → None

If the given path does not exist, creates a directory at it.

squirrel.fsspec.fs.get_fs_from_url(url: squirrel.constants.URL, **storage_options) → squirrel.constants.FILESYSTEM

Get filesystem suitable for a url.

Only the protocol part of the url is used, thus there is no need to call this method several times for different stores accessed with the same protocol. Should be called outside of store initialization to allow buffering from the same container among different stores or shards.

If the protocol “gs” is detected, will return a custom squirrel gcs file system instance which is more robust. For details, see squirrel.fsspec.custom_gcsfs.CustomGCSFileSystem).

squirrel.fsspec.fs.get_protocol(url: str) → str

Get the protocol from a url, return empty string if local