kedro.io.AbstractVersionedDataSet

AbstractVersionedDataSet is the base class for all versioned data set implementations. All data sets that implement versioning should extend this abstract class and implement the methods marked as abstract.

Example:

>>> from pathlib import Path, PurePosixPath
>>> import pandas as pd
>>> from kedro.io import AbstractVersionedDataSet
>>>
>>>
>>> class MyOwnDataSet(AbstractVersionedDataSet):
>>>     def __init__(self, filepath, version, param1, param2=True):
>>>         super().__init__(PurePosixPath(filepath), version)
>>>         self._param1 = param1
>>>         self._param2 = param2
>>>
>>>     def _load(self) -> pd.DataFrame:
>>>         load_path = self._get_load_path()
>>>         return pd.read_csv(load_path)
>>>
>>>     def _save(self, df: pd.DataFrame) -> None:
>>>         save_path = self._get_save_path()
>>>         df.to_csv(str(save_path))
>>>
>>>     def _exists(self) -> bool:
>>>         path = self._get_load_path()
>>>         return Path(path.as_posix()).exists()
>>>
>>>     def _describe(self):
>>>         return dict(version=self._version, param1=self._param1, param2=self._param2)

Example catalog.yml specification:

my_dataset:
    type: <path-to-my-own-dataset>.MyOwnDataSet
    filepath: data/01_raw/my_data.csv
    versioned: true
    param1: <param1-value> # param1 is a required argument
    # param2 will be True by default