squirrel.driver.jsonl
¶
Module Contents¶
Classes¶
A StoreDriver that by default uses SquirrelStore with jsonl serialization. |
-
class
squirrel.driver.jsonl.
JsonlDriver
(url: str, deser_hook: Optional[Callable] = None, **kwargs)¶ Bases:
squirrel.driver.store_driver.StoreDriver
A StoreDriver that by default uses SquirrelStore with jsonl serialization.
Initializes JsonlDriver with default store and serializer.
- Parameters
url (str) – Path to the root directory. If this path does not exist, it will be created.
deser_hook (Callable) – Callable that is passed as object_hook to
JsonDecoder
during json deserialization. Defaults to None.**kwargs – Keyword arguments passed to the super class initializer.
-
name
= jsonl¶
-
get_iter
(self, get_kwargs: Optional[Dict] = None, **kwargs) → squirrel.iterstream.Composable¶ Returns an iterable of samples as specified by fetcher_func.
- Parameters
get_kwargs (Dict) – Keyword arguments that will be passed as get_kwargs to
MapDriver.get_iter()
. get_kwargs will always have compression=”gzip”. Defaults to None.**kwargs – Other keyword arguments that will be passed to
MapDriver.get_iter()
.
- Returns
(Composable) Iterable over the items in the store.