kedro.config.ConfigLoader

class documentation

class ConfigLoader(AbstractConfigLoader): (source)

Recursively scan directories (config paths) contained in conf_source for configuration files with a yaml, yml, json, ini, pickle, xml or properties extension, load them, and return them in the form of a config dictionary.

The first processed config path is the base directory inside conf_source. The optional env argument can be used to specify a subdirectory of conf_source to process as a config path after base.

When the same top-level key appears in any 2 config files located in the same (sub)directory, a ValueError is raised.

When the same key appears in any 2 config files located in different (sub)directories, the last processed config path takes precedence and overrides this key.

For example, if your conf_source looks like this:

.
`-- conf
    |-- README.md
    |-- base
    |   |-- catalog.yml
    |   |-- logging.yml
    |   `-- experiment1
    |       `-- parameters.yml
    `-- local
        |-- catalog.yml
        |-- db.ini
        |-- experiment1
        |   |-- parameters.yml
        |   `-- model_parameters.yml
        `-- experiment2
            `-- parameters.yml

You can access the different configurations as follows:

>>> import logging.config
>>> from kedro.config import ConfigLoader
>>> from kedro.framework.project import settings
>>>
>>> conf_path = str(project_path / settings.CONF_SOURCE)
>>> conf_loader = ConfigLoader(conf_source=conf_path, env="local")
>>>
>>> conf_logging = conf_loader["logging"]
>>> logging.config.dictConfig(conf_logging)  # set logging conf
>>>
>>> conf_catalog = conf_loader["catalog"]
>>> conf_params = conf_loader["parameters"]

Method	`__getitem__`	Undocumented
Method	`__init__`	Instantiates a `ConfigLoader`.
Method	`__repr__`	Undocumented
Method	`get`	Undocumented
Instance Variable	`base_env`	Undocumented
Instance Variable	`config_patterns`	Undocumented
Instance Variable	`default_run_env`	Undocumented
Property	`conf_paths`	Property method to return deduplicated configuration paths.
Method	`_build_conf_paths`	Undocumented

Inherited from AbstractConfigLoader:

Instance Variable	`conf_source`	Undocumented
Instance Variable	`env`	Undocumented
Instance Variable	`runtime_params`	Undocumented

def __getitem__(self, key): (source) ¶

Undocumented

def __init__(self, conf_source: str, env: str = None, runtime_params: Dict[str, Any] = None, config_patterns: Dict[str, List[str]] = None, *, base_env: str = 'base', default_run_env: str = 'local'): (source) ¶

overrides kedro.config.AbstractConfigLoader.__init__

Instantiates a ConfigLoader.

Parameters
conf_source:`str`	Path to use as root directory for loading configuration.
env:`str`	Environment that will take precedence over base.
runtime_params:`Dict[str, Any]`	Extra parameters passed to a Kedro run.
config_patterns:`Dict[str, List[str]]`	Regex patterns that specify the naming convention for configuration files so they can be loaded. Can be customised by supplying config_patterns as in `CONFIG_LOADER_ARGS` in `settings.py`.
base_env:`str`	Name of the base environment. Defaults to `"base"`. This is used in the `conf_paths` property method to construct the configuration paths.
default_run_env:`str`	Name of the base environment. Defaults to `"local"`. This is used in the `conf_paths` property method to construct the configuration paths. Can be overriden by supplying the `env` argument.

def __repr__(self): (source) ¶

Undocumented

def get(self, *patterns: str) -> Dict[str, Any]: (source) ¶

Undocumented

base_env = (source) ¶

Undocumented

config_patterns: dict = (source) ¶

Undocumented

default_run_env = (source) ¶

Undocumented

@property
conf_paths = (source) ¶

Property method to return deduplicated configuration paths.

def _build_conf_paths(self) -> Iterable[str]: (source) ¶

Undocumented