kedro.config.common

module documentation

(source)

This module contains methods and facade interfaces for various ConfigLoader implementations.

Constant	`SUPPORTED_EXTENSIONS`	Undocumented
Function	`_check_duplicate_keys`	Undocumented
Function	`_get_config_from_patterns`	Recursively scan for configuration files, load and merge them, and return them in the form of a config dictionary.
Function	`_load_config_file`	Load an individual config file using `anyconfig` as a backend.
Function	`_load_configs`	Recursively load all configuration files, which satisfy a given list of glob patterns from a specific path.
Function	`_lookup_config_filepaths`	Undocumented
Function	`_path_lookup`	Return a set of all configuration files from `conf_path` or its subdirectories, which satisfy a given list of glob patterns.
Function	`_remove_duplicates`	Remove duplicates while preserving the order.
Variable	`_config_logger`	Undocumented

SUPPORTED_EXTENSIONS: list[str] = (source) ¶

Undocumented

Value

['.yml', '.yaml', '.json', '.ini', '.pickle', '.properties', '.xml']

def _check_duplicate_keys(processed_files: Dict[Path, AbstractSet[str]], filepath: Path, conf: Dict[str, Any]): (source) ¶

Undocumented

def _get_config_from_patterns(conf_paths: Iterable[str], patterns: Iterable[str] = None, ac_template: bool = False, ac_context: Dict[str, Any] = None) -> Dict[str, Any]: (source) ¶

Recursively scan for configuration files, load and merge them, and return them in the form of a config dictionary.

Parameters
conf_paths:`Iterable[str]`	List of configuration paths to directories
patterns:`Iterable[str]`	Glob patterns to match. Files, which names match any of the specified patterns, will be processed.
ac_template:`bool`	Boolean flag to indicate whether to use the `ac_template` argument of the `anyconfig.load` method. Used in the context of `_load_config_file` function.
ac_context:`Dict[str, Any]`	anyconfig context to pass to `anyconfig.load` method. Used in the context of `_load_config_file` function.
Returns
`Dict[str`, `Any]`	A Python dictionary with the combined configuration from all configuration files. Note: any keys that start with `_` will be ignored.
Raises
`ValueError`	If 2 or more configuration files inside the same config path (or its subdirectories) contain the same top-level key.
`MissingConfigException`	If no configuration files exist within a specified config path.
`BadConfigException`	If configuration is poorly formatted and cannot be loaded.

def _load_config_file(config_file: Path, ac_template: bool = False, ac_context: Dict[str, Any] = None) -> Dict[str, Any]: (source) ¶

Load an individual config file using anyconfig as a backend.

Parameters
config_file:`Path`	Path to a config file to process.
ac_template:`bool`	Boolean flag to indicate whether to use the `ac_template` argument of the `anyconfig.load` method.
ac_context:`Dict[str, Any]`	anyconfig context to pass to `anyconfig.load` method.
Returns
`Dict[str, Any]`	Parsed configuration.
Raises
`BadConfigException`	If configuration is poorly formatted and cannot be loaded.
`ParserError`	If file is invalid and cannot be parsed.

def _load_configs(config_filepaths: List[Path], ac_template: bool, ac_context: Dict[str, Any] = None) -> Dict[str, Any]: (source) ¶

Recursively load all configuration files, which satisfy a given list of glob patterns from a specific path.

Parameters
config_filepaths:`List[Path]`	Configuration files sorted in the order of precedence.
ac_template:`bool`	Boolean flag to indicate whether to use the `ac_template` argument of the `anyconfig.load` method. Used in the context of `_load_config_file` function.
ac_context:`Dict[str, Any]`	anyconfig context to pass to `anyconfig.load` method. Used in the context of `_load_config_file` function.
Returns
`Dict[str, Any]`	Resulting configuration dictionary.
Raises
`ValueError`	If 2 or more configuration files contain the same key(s).
`BadConfigException`	If configuration is poorly formatted and cannot be loaded.

def _lookup_config_filepaths(conf_path: Path, patterns: Iterable[str], processed_files: Set[Path], logger: Any) -> List[Path]: (source) ¶

Undocumented

def _path_lookup(conf_path: Path, patterns: Iterable[str]) -> Set[Path]: (source) ¶

Return a set of all configuration files from conf_path or its subdirectories, which satisfy a given list of glob patterns.

Parameters
conf_path:`Path`	Path to configuration directory.
patterns:`Iterable[str]`	List of glob patterns to match the filenames against.
Returns
`Set[Path]`	A set of paths to configuration files.

def _remove_duplicates(items: Iterable[str]): (source) ¶

Remove duplicates while preserving the order.

Undocumented