Using the PyTask

The PyTask allows you to call any Python function. That function can (optionally) return a FWAction to perform dynamic actions (e.g., see the Guide to Writing Firetasks). Thus, with the PyTask you can basically do anything!

Required parameters

• func (str): a fully qualified python method. If the function is part of a module then the name specification is something like module.function, e.g. json.dump, or shutil.copy, or a user function.

Optional parameters

• args (list): a list of positional arguments to feed into the function. Default is an empty list.

• kwargs (dict): a dictionary of keyword arguments. Default is empty.

• auto_kwargs (bool): If True, all other parameters not starting with “_” are automatically supplied as keyword args.

• stored_data_varname (str): If this is a string that does not evaluate to False, the output of the function will be stored as FWAction(stored_data={stored_data_varname: output}). The name is deliberately long to avoid potential name conflicts.

• inputs (list): a list of spec keys which will be used to pass data from spec to the function as positional arguments; the so generated arguments list will be appended to args. Default is an empty list.

• outputs (list): a list of spec keys that will be used to pass returned values from the function to the spec of current and of child Fireworks; If the list is empty ([]) or not specified (None) then the output of the function will not be stored/passed. Default is an empty list.

• chunk_number (int): a serial number of the Firetask within a group of Firetasks generated by a ForeachTask. If chunk_number is not None the output will be merged with the output of the other Firetasks in the group into a list under the key specified in outputs. Default is None.

Examples

Example 1: Using PyTask with static arguments

Here is an example of defining a PyTask that sleeps for 5 seconds:

fw_timer = Firework(PyTask(func='time.sleep',args=[5]))

Note that you can call any Python function this way!

Example 2: Redirecting data from and to spec

Here is an example of using PyTask in a dataflow context:

fws:
- fw_id: 1
  name: Grind coffee
  spec:
    _tasks:
    - _fw_name: PyTask
      func: auxiliary.printurn
      inputs: [coffee beans]
      outputs: [coffee powder]
    coffee beans: best selection
- fw_id: 2
  name: Brew coffee
  spec:
    _tasks:
    - _fw_name: PyTask
      func: auxiliary.printurn
      inputs: [coffee powder, water]
      outputs: [pure coffee]
    water: workflowing water
links:
  '1': [2]
metadata: {}
name: Simple coffee workflow

In this example the function auxiliary.printurn prints and returns all its arguments:

def printurn(*args):
    result = []
    for arg in args:
        if isinstance(arg, list) and len(arg) == 1:
            result.append(arg[0])
        else:
            result.append(arg)
    if len(result) == 1:
        result = result[0]
    print(result)
    return result

The module auxiliary, i.e. the file auxiliary.py must be in $PYTHONPATH.