jobflow.managers

jobflow.managers.local

Tools for running jobflow locally.

jobflow.managers.local.run_locally(flow, log=True, store=None, create_folders=False, root_dir=None, ensure_success=False, allow_external_references=False, raise_immediately=False)[source]

Run a Job or Flow locally.

Parameters:

  • flow (Flow | Job | list[Job]) – A job or flow.

  • log (bool | str) – Controls logging. Defaults to True. Can be: - False: disable logging - True: use default logging format (read from ~/.jobflow.yaml) - str: custom logging format string (e.g. “%(message)s” for more concise output)

  • store (JobStore) – A job store. If a job store is not specified then JobflowSettings.JOB_STORE will be used. By default this is a maggma MemoryStore but can be customised by setting the jobflow configuration file.

  • create_folders (bool) – Whether to run each job in a new folder.

  • root_dir (str | Path | None) –

    The root directory to run the jobs in or where to create new subfolders if

    create_folders is True. If None then the current working directory will be used.

  • ensure_success (bool) – Raise an error if the flow was not executed successfully.

  • allow_external_references (bool) – If False all the references to other outputs should be from other Jobs of the same Flow.

  • raise_immediately (bool) – If True, raise an exception immediately if a job fails. If False, continue running the flow and only raise an exception at the end if the flow did not finish running successfully.

Returns:

The responses of the jobs, as a dict of {uuid: {index: response}}.

Return type:

dict[str, dict[int, Response]]

jobflow.managers.fireworks

Tools for running Flow and Job objects using the FireWorks package.

jobflow.managers.fireworks.flow_to_workflow(flow, store=None, allow_external_references=False, **kwargs)[source]

Convert a Flow or a Job to a FireWorks Workflow object.

Each firework spec is updated with the contents of the Job.config.manager_config dictionary. Accordingly, a JobConfig object can be used to configure FireWork options such as metadata and the fireworker.

Parameters:

  • flow (jobflow.Flow | jobflow.Job | list[jobflow.Job]) – A flow or job.

  • store (jobflow.JobStore) – A job store. Alternatively, if set to None, JobflowSettings.JOB_STORE will be used. Note, this could be different on the computer that submits the workflow and the computer which runs the workflow. The value of JOB_STORE on the computer that runs the workflow will be used.

  • allow_external_references (bool) – If False all the references to other outputs should be from other Jobs of the Flow.

  • **kwargs – Keyword arguments passed to Workflow init method.

Returns:

The job or flow as a workflow.

Return type:

Workflow

jobflow.managers.fireworks.job_to_firework(job, store=None, parents=None, parent_mapping=None, **kwargs)[source]

Convert a Job to a Firework.

The firework spec is updated with the contents of the Job.config.manager_config dictionary. Accordingly, a JobConfig object can be used to configure FireWork options such as metadata and the fireworker.

Parameters:

  • job (jobflow.Job) – A job.

  • store (jobflow.JobStore) – A job store. Alternatively, if set to None, JobflowSettings.JOB_STORE will be used. Note, this could be different on the computer that submits the workflow and the computer which runs the workflow. The value of JOB_STORE on the computer that runs the workflow will be used.

  • parents (Sequence[str]) – The parent uuids of the job.

  • parent_mapping (dict[str, Firework]) – A dictionary mapping job uuids to Firework objects, as {uuid: Firework}.

  • **kwargs – Keyword arguments passed to the Firework constructor.

Returns:

A firework that will run the job.

Return type:

Firework

class jobflow.managers.fireworks.JobFiretask(*args, **kwargs)[source]

Bases: FiretaskBase

A firetask that will run any job.

Parameters:

  • job (Dict) – A serialized job.

  • store (JobStore) – A job store. Alternatively, if set to None, JobflowSettings.JOB_STORE will be used. Note, this could be different on the computer that submits the workflow and the computer which runs the workflow. The value of JOB_STORE on the computer that runs the workflow will be used.

run_task(fw_spec)[source]

Run the job and handle any dynamic firework submissions.