This page is available as a Jupyter notebook: tutorials/3-defining-jobs.ipynb.

Defining jobs in jobflow#

In this tutorial, you will:

  • Learn about the job decorator.

  • Understand the structure of the Job object.

  • Set the configuration settings of a job.

  • Use the Response object.

  • Learn tips for writing job functions.

The purpose of this tutorial is to delve into the basic functionality of jobs and gain a feeling for what is possible. Later tutorials will describe how to employ jobs in complex workflows.

Creating job objects#

The building block of jobflows are Job objects. Jobs are delayed calls to python functions whose outputs are stored in a database. The easiest way to create a job is using the @job decorator. The job decorator can be applied to any function, even those with optional parameters.

[2]:

from jobflow import job


@job
def add(a, b, c=2):
    return a + b + c

Any call to the add function will return a Job object.

[12]:

add_first = add(1, 2, c=5)

Each job is assigned a unique identifier (UUID).

[13]:

add_first.uuid

[13]:

'30bc700e-dfe0-4a3f-aec9-81ef54446e92'

Jobs also have an index. This tracks the number of times the job has been “replaced” (replacing is covered in detail in the Dynamic and nested Flows tutorial).

[14]:

add_first.index

[14]:

1

Jobs have outputs that can be accessed using the output attribute. As the job has not yet been executed, the output is currently a reference to the future output.

[15]:

add_first.output

[15]:

OutputReference(30bc700e-dfe0-4a3f-aec9-81ef54446e92)

The output of a job can be used as the input to another job.

[16]:

add_second = add(add_first.output, 3)

The output does not have to be an argument on its own, it can be included in a list or a dictionary.

[18]:

@job
def sum_numbers(numbers):
    return sum(numbers)


sum_job = sum_numbers([add_first.output, add_second.output])