class documentation

class SequentialRunner(AbstractRunner): (source)

View In Hierarchy

SequentialRunner is an AbstractRunner implementation. It can be used to run the Pipeline in a sequential manner using a topological sort of provided nodes.

Method __init__ Instantiates the runner classs.
Method create_default_data_set Factory method for creating the default data set for the runner.
Method _run The method implementing sequential pipeline running.

Inherited from AbstractRunner:

Method run Run the Pipeline using the datasets provided by catalog and save results back to the same objects.
Method run_only_missing Run only the missing outputs from the Pipeline using the datasets provided by catalog, and save results back to the same objects.
Method _suggest_resume_scenario Suggest a command to the user to resume a run after it fails. The run should be started from the point closest to the failure for which persisted input exists.
Instance Variable _is_async Undocumented
Property _logger Undocumented
def __init__(self, is_async: bool = False): (source)

Instantiates the runner classs.

Parameters
is_async:boolIf True, the node inputs and outputs are loaded and saved asynchronously with threads. Defaults to False.
def create_default_data_set(self, ds_name: str) -> AbstractDataSet: (source)

Factory method for creating the default data set for the runner.

Parameters
ds_name:strName of the missing data set
Returns
AbstractDataSetAn instance of an implementation of AbstractDataSet to be used for all unregistered data sets.
def _run(self, pipeline: Pipeline, catalog: DataCatalog, hook_manager: PluginManager, session_id: str = None): (source)

The method implementing sequential pipeline running.

Parameters
pipeline:PipelineThe Pipeline to run.
catalog:DataCatalogThe DataCatalog from which to fetch data.
hook_manager:PluginManagerThe PluginManager to activate hooks.
session_id:strThe id of the session.
Raises
Exceptionin case of any downstream node failure.