kedro.io.data_catalog

module documentation
(source)

DataCatalog stores instances of AbstractDataSet implementations to provide load and save capabilities from anywhere in the program. To use a DataCatalog, you need to instantiate it with a dictionary of data sets. Then it will act as a single point of reference for your calls, relaying load and save functions to the underlying data sets.

Constant CATALOG_KEY Undocumented
Constant CREDENTIALS_KEY Undocumented
Constant WORDS_REGEX_PATTERN Undocumented
Class _FrozenDatasets Helper class to access underlying loaded datasets
Function _get_credentials Return a set of credentials from the provided credentials dict.
Function _resolve_credentials Return the dataset configuration where credentials are resolved using credentials dictionary provided.
Function _sub_nonword_chars Replace non-word characters in data set names since Kedro 0.16.2.
CATALOG_KEY: str = (source)

Undocumented

Value
'catalog'
CREDENTIALS_KEY: str = (source)

Undocumented

Value
'credentials'
WORDS_REGEX_PATTERN = (source)

Undocumented

Value
re.compile(r'\W+')
def _get_credentials(credentials_name: str, credentials: Dict[str, Any]) -> Dict[str, Any]: (source)

Return a set of credentials from the provided credentials dict.

Parameters
credentials_name:strCredentials name.
credentials:Dict[str, Any]A dictionary with all credentials.
Returns
Dict[str, Any]The set of requested credentials.
Raises
KeyErrorWhen a data set with the given name has not yet been registered.
def _resolve_credentials(config: Dict[str, Any], credentials: Dict[str, Any]) -> Dict[str, Any]: (source)

Return the dataset configuration where credentials are resolved using credentials dictionary provided.

Parameters
config:Dict[str, Any]Original dataset config, which may contain unresolved credentials.
credentials:Dict[str, Any]A dictionary with all credentials.
Returns
Dict[str, Any]The dataset config, where all the credentials are successfully resolved.
def _sub_nonword_chars(data_set_name: str) -> str: (source)

Replace non-word characters in data set names since Kedro 0.16.2.

Parameters
data_set_name:strThe data set name registered in the data catalog.
Returns
strThe name used in DataCatalog.datasets.