squirrel.store.squirrel_store
¶
Module Contents¶
Classes¶
FilesystemStore that persist samples (Dict objects) or shards (i.e. list of samples). |
-
class
squirrel.store.squirrel_store.
SquirrelStore
(url: str, serializer: squirrel.serialization.SquirrelSerializer, clean: bool = False, **storage_options)¶ Bases:
squirrel.store.filesystem.FilesystemStore
FilesystemStore that persist samples (Dict objects) or shards (i.e. list of samples).
Initializes SquirrelStore.
- Parameters
url (str) – Path to the root directory. If this path does not exist, it will be created.
serializer (SquirrelSerializer) – Serializer that is used to serialize data before persisting (see
set()
) and to deserialize data after reading (seeget()
). If not specified, data will not be (de)serialized. Defaults to None.clean (bool) – If true, all files in the store will be removed recursively
**storage_options – Keyword arguments passed to filesystem initializer.
-
get
(self, key: str, **kwargs) → Iterator[squirrel.constants.SampleType]¶ Yields the item with the given key.
If the store has a serializer, data read from the file will be deserialized.
- Parameters
key (str) – Key corresponding to the item to retrieve.
**kwargs – Keyword arguments forwarded to
self.serializer.deserialize_shard_from_file()
.
- Yields
(Any) Item with the given key.
-
set
(self, value: Union[squirrel.constants.SampleType, squirrel.constants.ShardType], key: Optional[str] = None, **kwargs) → None¶ Persists a shard or sample with the given key.
Data item will be serialized before writing to a file.
- Parameters
value (Any) – Shard or sample to be persisted. If value is a sample (i.e. not a list), it will be wrapped around with a list before persisting.
key (Optional[str]) – Optional key corresponding to the item to persist.
**kwargs – Keyword arguments forwarded to
self.serializer.serialize_shard_to_file()
.