fireworks.user_objects.firetasks package

Submodules

fireworks.user_objects.firetasks.dataflow_tasks module

This module includes dataflow firetask tasks

class fireworks.user_objects.firetasks.dataflow_tasks.CommandLineTask(*args, **kwargs)

Bases: fireworks.core.firework.FiretaskBase

A Firetask to execute external commands in a shell

Required params:

  • command_spec (dict): a dictionary specification of the command (see below for details)

Optional params:

  • inputs ([str]): list of labels, one for each input argument

  • outputs ([str]): list of labels, one for each output argument

  • chunk_number (int): the serial number of the firetask when it is part of a series generated by a ForeachTask

command_spec = {

‘command’: [str], # mandatory label_1: dict_1, # optional label_2: dict_2, # optional …

} The ‘command’ is a representation of the command as to be used with subprocess package. The optional keys label_1, label_2, etc. are the actual labels used in the inputs and outputs. The dictionaries dict_1, dict_2, etc. have the following schema: {

‘binding’: {

prefix: str or None, separator: str or None

}, ‘source’: {

‘type’: ‘path’ or ‘data’ or ‘identifier’

or ‘stdin’ or ‘stdout’ or ‘stderr’ or None,

‘value’: str or int or float

}, ‘target’: {

‘type’: ‘path’ or ‘data’ or ‘identifier’

or ‘stdin’ or ‘stdout’ or ‘stderr’ or None,

‘value’: str

}

}

  • If the ‘type’ in the ‘source’ field is ‘data’ the ‘value’ can be of

types ‘str’, ‘int’ and ‘float’.

  • When a str is found instead of dict for some ‘source’, for example

‘source’: ‘string’, ‘string’ is replaced with spec[‘string’] which must be available and of the schema of the ‘source’.

  • When a str is found instead of dict for some label, for example

label: ‘string’, ‘string’ is replaced with spec[‘string’] which can be a dictionary with this schema or a list of such dictionaries.

static command_line_tool(command, inputs=None, outputs=None)

This function composes and executes a command from provided specifications.

Required parameters:

  • command ([str]): the command as to be passed to subprocess.Popen

Optional parameters:

  • inputs ([dict, [dict]]): list of the specifications for inputs; multiple inputs may be passed in one list of dictionaries

  • outputs ([dict]): list of the specifications for outputs

Returns

‘target’: {
‘type’: ‘path’ or ‘data’ or ‘identifier’

or ‘stdin’ or ‘stdout’ or ‘stderr’ or None

’value’: str

}

If outputs is None then an empty list is returned.

Return type

  • list of target dictionaries for each output

optional_params = ['inputs', 'outputs', 'chunk_number']
required_params = ['command_spec']
run_task(fw_spec)

This method gets called when the Firetask is run. It can take in a Firework spec, perform some task using that data, and then return an output in the form of a FWAction.

Parameters

fw_spec (dict) – A Firework spec. This comes from the master spec. In addition, this spec contains a special “_fw_env” key that contains the env settings of the FWorker calling this method. This provides for abstracting out certain commands or settings. For example, “foo” may be named “foo1” in resource 1 and “foo2” in resource 2. The FWorker env can specify { “foo”: “foo1”}, which maps an abstract variable “foo” to the relevant “foo1” or “foo2”. You can then write a task that uses fw_spec[“_fw_env”][“foo”] that will work across all these multiple resources.

Returns

(FWAction)

class fireworks.user_objects.firetasks.dataflow_tasks.ForeachTask(*args, **kwargs)

Bases: fireworks.core.firework.FiretaskBase

This firetask branches the workflow creating parallel fireworks using FWAction: one firework for each element or each chunk from the split list. Each firework in this generated list contains the firetask specified in the task dictionary. If the number of chunks is specified the split list will be divided into this number of chunks and each chunk will be processed by one of the generated child fireworks.

Required params:

  • task (dict): a dictionary version of the firetask

  • split (str): a label of an input list; it must be available both in the inputs list of the specified task and in the spec.

Optional params:

  • number of chunks (int): if provided the split input list will be divided into this number of sublists and each will be processed by a separate child firework

optional_params = ['number of chunks']
required_params = ['task', 'split']
run_task(fw_spec)

This method gets called when the Firetask is run. It can take in a Firework spec, perform some task using that data, and then return an output in the form of a FWAction.

Parameters

fw_spec (dict) – A Firework spec. This comes from the master spec. In addition, this spec contains a special “_fw_env” key that contains the env settings of the FWorker calling this method. This provides for abstracting out certain commands or settings. For example, “foo” may be named “foo1” in resource 1 and “foo2” in resource 2. The FWorker env can specify { “foo”: “foo1”}, which maps an abstract variable “foo” to the relevant “foo1” or “foo2”. You can then write a task that uses fw_spec[“_fw_env”][“foo”] that will work across all these multiple resources.

Returns

(FWAction)

class fireworks.user_objects.firetasks.dataflow_tasks.ImportDataTask(*args, **kwargs)

Bases: fireworks.core.firework.FiretaskBase

Update the spec with data from file in a nested dictionary at a position specified by a mapstring = maplist[0]/maplist[1]/… i.e. spec[maplist[0]][maplist[1]]… = data

optional_params = []
required_params = ['filename', 'mapstring']
run_task(fw_spec)

This method gets called when the Firetask is run. It can take in a Firework spec, perform some task using that data, and then return an output in the form of a FWAction.

Parameters

fw_spec (dict) – A Firework spec. This comes from the master spec. In addition, this spec contains a special “_fw_env” key that contains the env settings of the FWorker calling this method. This provides for abstracting out certain commands or settings. For example, “foo” may be named “foo1” in resource 1 and “foo2” in resource 2. The FWorker env can specify { “foo”: “foo1”}, which maps an abstract variable “foo” to the relevant “foo1” or “foo2”. You can then write a task that uses fw_spec[“_fw_env”][“foo”] that will work across all these multiple resources.

Returns

(FWAction)

class fireworks.user_objects.firetasks.dataflow_tasks.JoinDictTask(*args, **kwargs)

Bases: fireworks.core.firework.FiretaskBase

combines specified spec fields into a dictionary

optional_params = ['rename']
required_params = ['inputs', 'output']
run_task(fw_spec)

This method gets called when the Firetask is run. It can take in a Firework spec, perform some task using that data, and then return an output in the form of a FWAction.

Parameters

fw_spec (dict) – A Firework spec. This comes from the master spec. In addition, this spec contains a special “_fw_env” key that contains the env settings of the FWorker calling this method. This provides for abstracting out certain commands or settings. For example, “foo” may be named “foo1” in resource 1 and “foo2” in resource 2. The FWorker env can specify { “foo”: “foo1”}, which maps an abstract variable “foo” to the relevant “foo1” or “foo2”. You can then write a task that uses fw_spec[“_fw_env”][“foo”] that will work across all these multiple resources.

Returns

(FWAction)

class fireworks.user_objects.firetasks.dataflow_tasks.JoinListTask(*args, **kwargs)

Bases: fireworks.core.firework.FiretaskBase

combines specified spec fields into a list.

required_params = ['inputs', 'output']
run_task(fw_spec)

This method gets called when the Firetask is run. It can take in a Firework spec, perform some task using that data, and then return an output in the form of a FWAction.

Parameters

fw_spec (dict) – A Firework spec. This comes from the master spec. In addition, this spec contains a special “_fw_env” key that contains the env settings of the FWorker calling this method. This provides for abstracting out certain commands or settings. For example, “foo” may be named “foo1” in resource 1 and “foo2” in resource 2. The FWorker env can specify { “foo”: “foo1”}, which maps an abstract variable “foo” to the relevant “foo1” or “foo2”. You can then write a task that uses fw_spec[“_fw_env”][“foo”] that will work across all these multiple resources.

Returns

(FWAction)

fireworks.user_objects.firetasks.fileio_tasks module

class fireworks.user_objects.firetasks.fileio_tasks.ArchiveDirTask(*args, **kwargs)

Bases: fireworks.core.firework.FiretaskBase

Wrapper around shutil.make_archive to make tar archives.

Parameters

  • base_name (str) – Name of the file to create, including the path, minus any format-specific extension.

  • format (str) – Optional. one of “zip”, “tar”, “bztar” or “gztar”. Defaults to gztar.

optional_params = ['format']
required_params = ['base_name']
run_task(fw_spec)

This method gets called when the Firetask is run. It can take in a Firework spec, perform some task using that data, and then return an output in the form of a FWAction.

Parameters

fw_spec (dict) – A Firework spec. This comes from the master spec. In addition, this spec contains a special “_fw_env” key that contains the env settings of the FWorker calling this method. This provides for abstracting out certain commands or settings. For example, “foo” may be named “foo1” in resource 1 and “foo2” in resource 2. The FWorker env can specify { “foo”: “foo1”}, which maps an abstract variable “foo” to the relevant “foo1” or “foo2”. You can then write a task that uses fw_spec[“_fw_env”][“foo”] that will work across all these multiple resources.

Returns

(FWAction)

class fireworks.user_objects.firetasks.fileio_tasks.CompressDirTask(*args, **kwargs)

Bases: fireworks.core.firework.FiretaskBase

Compress all files in a directory.

Parameters

  • dest (str) – Optional. Path to compress.

  • compression (str) – Optional. Can only be gz or bz2. Defaults to gz.

  • ignore_errors (bool) – Optional. Whether to ignore errors. Defaults to False.

optional_params = ['compression', 'dest', 'ignore_errors']
run_task(fw_spec)

This method gets called when the Firetask is run. It can take in a Firework spec, perform some task using that data, and then return an output in the form of a FWAction.

Parameters

fw_spec (dict) – A Firework spec. This comes from the master spec. In addition, this spec contains a special “_fw_env” key that contains the env settings of the FWorker calling this method. This provides for abstracting out certain commands or settings. For example, “foo” may be named “foo1” in resource 1 and “foo2” in resource 2. The FWorker env can specify { “foo”: “foo1”}, which maps an abstract variable “foo” to the relevant “foo1” or “foo2”. You can then write a task that uses fw_spec[“_fw_env”][“foo”] that will work across all these multiple resources.

Returns

(FWAction)

class fireworks.user_objects.firetasks.fileio_tasks.DecompressDirTask(*args, **kwargs)

Bases: fireworks.core.firework.FiretaskBase

Decompress all files in a directory. Autodetects gz, bz2 and z file extensions.

Parameters

  • dest (str) – Optional. Path to decompress.

  • ignore_errors (bool) – Optional. Whether to ignore errors. Defaults to False.

optional_params = ['dest', 'ignore_errors']
run_task(fw_spec)

This method gets called when the Firetask is run. It can take in a Firework spec, perform some task using that data, and then return an output in the form of a FWAction.

Parameters

fw_spec (dict) – A Firework spec. This comes from the master spec. In addition, this spec contains a special “_fw_env” key that contains the env settings of the FWorker calling this method. This provides for abstracting out certain commands or settings. For example, “foo” may be named “foo1” in resource 1 and “foo2” in resource 2. The FWorker env can specify { “foo”: “foo1”}, which maps an abstract variable “foo” to the relevant “foo1” or “foo2”. You can then write a task that uses fw_spec[“_fw_env”][“foo”] that will work across all these multiple resources.

Returns

(FWAction)

class fireworks.user_objects.firetasks.fileio_tasks.FileDeleteTask(*args, **kwargs)

Bases: fireworks.core.firework.FiretaskBase

A Firetask to delete files:

Required params:

  • files_to_delete: ([str]) Filenames to delete

Optional params:

  • dest: (str) Shared path for files

  • ignore_errors (bool): Whether to ignore errors. Defaults to True.

optional_params = ['dest', 'ignore_errors']
required_params = ['files_to_delete']
run_task(fw_spec)

This method gets called when the Firetask is run. It can take in a Firework spec, perform some task using that data, and then return an output in the form of a FWAction.

Parameters

fw_spec (dict) – A Firework spec. This comes from the master spec. In addition, this spec contains a special “_fw_env” key that contains the env settings of the FWorker calling this method. This provides for abstracting out certain commands or settings. For example, “foo” may be named “foo1” in resource 1 and “foo2” in resource 2. The FWorker env can specify { “foo”: “foo1”}, which maps an abstract variable “foo” to the relevant “foo1” or “foo2”. You can then write a task that uses fw_spec[“_fw_env”][“foo”] that will work across all these multiple resources.

Returns

(FWAction)

class fireworks.user_objects.firetasks.fileio_tasks.FileTransferTask(*args, **kwargs)

Bases: fireworks.core.firework.FiretaskBase

A Firetask to Transfer files. Note that

Required params:

  • mode: (str) - move, mv, copy, cp, copy2, copytree, copyfile, rtransfer

  • files: ([str]) or ([(str, str)]) - list of source files, or dictionary containing

    ‘src’ and ‘dest’ keys

Optional params:

  • dest: (str) destination directory, if not specified within files parameter (else optional)

  • server: (str) server host for remote transfer

  • user: (str) user to authenticate with on remote server

  • key_filename: (str) optional SSH key location for remote transfer

  • max_retry: (int) number of times to retry failed transfers; defaults to 0 (no retries)

  • retry_delay: (int) number of seconds to wait between retries; defaults to 10

  • ignore_errors (bool): Optional. Whether to ignore errors. Defaults to False.

fn_list = {'copy': <function copy>, 'copy2': <function copy2>, 'copyfile': <function copyfile>, 'copytree': <function copytree>, 'cp': <function copy>, 'move': <function move>, 'mv': <function move>}
optional_params = ['dest', 'server', 'user', 'key_filename', 'max_retry', 'retry_delay', 'ignore_errors']
required_params = ['mode', 'files']
run_task(fw_spec)

This method gets called when the Firetask is run. It can take in a Firework spec, perform some task using that data, and then return an output in the form of a FWAction.

Parameters

fw_spec (dict) – A Firework spec. This comes from the master spec. In addition, this spec contains a special “_fw_env” key that contains the env settings of the FWorker calling this method. This provides for abstracting out certain commands or settings. For example, “foo” may be named “foo1” in resource 1 and “foo2” in resource 2. The FWorker env can specify { “foo”: “foo1”}, which maps an abstract variable “foo” to the relevant “foo1” or “foo2”. You can then write a task that uses fw_spec[“_fw_env”][“foo”] that will work across all these multiple resources.

Returns

(FWAction)

class fireworks.user_objects.firetasks.fileio_tasks.FileWriteTask(*args, **kwargs)

Bases: fireworks.core.firework.FiretaskBase

A Firetask to write files:

Required params:
  • files_to_write: ([{filename:(str), contents:(str)}]) List of dicts with filenames

    and contents

Optional params:

  • dest: (str) Shared path for files

optional_params = ['dest']
required_params = ['files_to_write']
run_task(fw_spec)

This method gets called when the Firetask is run. It can take in a Firework spec, perform some task using that data, and then return an output in the form of a FWAction.

Parameters

fw_spec (dict) – A Firework spec. This comes from the master spec. In addition, this spec contains a special “_fw_env” key that contains the env settings of the FWorker calling this method. This provides for abstracting out certain commands or settings. For example, “foo” may be named “foo1” in resource 1 and “foo2” in resource 2. The FWorker env can specify { “foo”: “foo1”}, which maps an abstract variable “foo” to the relevant “foo1” or “foo2”. You can then write a task that uses fw_spec[“_fw_env”][“foo”] that will work across all these multiple resources.

Returns

(FWAction)

fireworks.user_objects.firetasks.filepad_tasks module

class fireworks.user_objects.firetasks.filepad_tasks.AddFilesTask(*args, **kwargs)

Bases: fireworks.core.firework.FiretaskBase

A Firetask to add files to the filepad.

Required params:

  • paths (list/str): either list of paths or a glob pattern string.

Optional params:

  • identifiers ([str]): list of identifiers, one for each file in the paths list

  • directory (str): path to directory where the pattern matching is to be done.

  • filepad_file (str): path to the filepad db config file

  • compress (bool): whether or not to compress the file before inserting to gridfs

  • metadata (dict): metadata to store along with the file, stored in ‘metadata’ key

optional_params = ['identifiers', 'directory', 'filepad_file', 'compress', 'metadata']
required_params = ['paths']
run_task(fw_spec)

This method gets called when the Firetask is run. It can take in a Firework spec, perform some task using that data, and then return an output in the form of a FWAction.

Parameters

fw_spec (dict) – A Firework spec. This comes from the master spec. In addition, this spec contains a special “_fw_env” key that contains the env settings of the FWorker calling this method. This provides for abstracting out certain commands or settings. For example, “foo” may be named “foo1” in resource 1 and “foo2” in resource 2. The FWorker env can specify { “foo”: “foo1”}, which maps an abstract variable “foo” to the relevant “foo1” or “foo2”. You can then write a task that uses fw_spec[“_fw_env”][“foo”] that will work across all these multiple resources.

Returns

(FWAction)

class fireworks.user_objects.firetasks.filepad_tasks.DeleteFilesTask(*args, **kwargs)

Bases: fireworks.core.firework.FiretaskBase

A Firetask to delete files from the filepad

Required params:

  • identifiers ([str]): identifiers of files to delete

Optional params:

  • filepad_file (str): path to the filepad db config file

optional_params = ['filepad_file']
required_params = ['identifiers']
run_task(fw_spec)

This method gets called when the Firetask is run. It can take in a Firework spec, perform some task using that data, and then return an output in the form of a FWAction.

Parameters

fw_spec (dict) – A Firework spec. This comes from the master spec. In addition, this spec contains a special “_fw_env” key that contains the env settings of the FWorker calling this method. This provides for abstracting out certain commands or settings. For example, “foo” may be named “foo1” in resource 1 and “foo2” in resource 2. The FWorker env can specify { “foo”: “foo1”}, which maps an abstract variable “foo” to the relevant “foo1” or “foo2”. You can then write a task that uses fw_spec[“_fw_env”][“foo”] that will work across all these multiple resources.

Returns

(FWAction)

class fireworks.user_objects.firetasks.filepad_tasks.GetFilesByQueryTask(*args, **kwargs)

Bases: fireworks.core.firework.FiretaskBase

A Firetask to query files from the filepad and write them to specified directory (current working directory if not specified).

Required params:

  • query (dict): mongo db query identifying files to fetch. Same as within fireworks.utilities.dict_mods, use ‘->’ in dict keys for querying nested documents, instead of MongoDB ‘.’ (dot) separator. Do use ‘.’ and NOT ‘->’ within the ‘sort_key’ field.

Optional params:

  • sort_key (str): sort key, don’t sort per default

  • sort_direction (int): sort direction, default ‘pymongo.DESCENDING’

  • limit (int): maximum number of files to write, default: no limit

  • fizzle_empty_result (bool): fizzle if no file found, default: True

  • fizzle_degenerate_file_name (bool): fizzle if more than one of the resulting files are to be written to the same local file (i.e. the filepad’s ‘original_file_name’ entries overlap), default: True

  • filepad_file (str): path to the filepad db config file

  • dest_dir (str): destination directory, default is the current working directory.

  • new_file_names ([str]): if provided, the retrieved files will be renamed. Not recommended as order and number of queried files not fixed.

  • meta_file (bool): default: False. If True, then metadata of queried files written to a .yaml file of same name as file itself, suffixed by…

  • meta_file_suffix (str): if not None, metadata for each file is written to a YAML file of the same name, suffixed by this string. Default: “.meta.yaml”

The options ‘fizzle_degenerate_file_name’, ‘limit’, ‘sort_key’, and ‘sort_direction’ are all inntended to help dealing with the following special case: Querying by metadata leads to an a priori unknown number of files in the general case. Thus, it is advisable to either ‘limit’ the number of files and/or avoid explicitly specifying a list of ‘new_file_names’. In the latter case, files will be written to ‘dest_dir’ using their ‘original_file_name’ recorded within the attached FilePad object. When more than one queried file share the same ‘original_file_name’, the order of processing matters: subsequent files will overwrite their predecessor of same name. ‘sort_key’ and ‘sort_direction’ can help to assure deterministic behavior, e.g. by always processing newer files later.

optional_params = ['dest_dir', 'filepad_file', 'fizzle_degenerate_file_name', 'fizzle_empty_result', 'limit', 'meta_file', 'meta_file_suffix', 'new_file_names', 'sort_direction', 'sort_key']
required_params = ['query']
run_task(fw_spec)

This method gets called when the Firetask is run. It can take in a Firework spec, perform some task using that data, and then return an output in the form of a FWAction.

Parameters

fw_spec (dict) – A Firework spec. This comes from the master spec. In addition, this spec contains a special “_fw_env” key that contains the env settings of the FWorker calling this method. This provides for abstracting out certain commands or settings. For example, “foo” may be named “foo1” in resource 1 and “foo2” in resource 2. The FWorker env can specify { “foo”: “foo1”}, which maps an abstract variable “foo” to the relevant “foo1” or “foo2”. You can then write a task that uses fw_spec[“_fw_env”][“foo”] that will work across all these multiple resources.

Returns

(FWAction)

class fireworks.user_objects.firetasks.filepad_tasks.GetFilesTask(*args, **kwargs)

Bases: fireworks.core.firework.FiretaskBase

A Firetask to fetch files from the filepad and write it to specified directory (current working directory if not specified)

Required params:

  • identifiers ([str]): identifiers of files to fetch

Optional params:

  • filepad_file (str): path to the filepad db config file

  • dest_dir (str): destination directory, default is the current working directory

  • new_file_names ([str]): if provided, the retrieved files will be renamed

optional_params = ['filepad_file', 'dest_dir', 'new_file_names']
required_params = ['identifiers']
run_task(fw_spec)

This method gets called when the Firetask is run. It can take in a Firework spec, perform some task using that data, and then return an output in the form of a FWAction.

Parameters

fw_spec (dict) – A Firework spec. This comes from the master spec. In addition, this spec contains a special “_fw_env” key that contains the env settings of the FWorker calling this method. This provides for abstracting out certain commands or settings. For example, “foo” may be named “foo1” in resource 1 and “foo2” in resource 2. The FWorker env can specify { “foo”: “foo1”}, which maps an abstract variable “foo” to the relevant “foo1” or “foo2”. You can then write a task that uses fw_spec[“_fw_env”][“foo”] that will work across all these multiple resources.

Returns

(FWAction)

fireworks.user_objects.firetasks.filepad_tasks.get_fpad(fpad_file)

fireworks.user_objects.firetasks.script_task module

This module includes tasks to integrate scripts and python functions

class fireworks.user_objects.firetasks.script_task.PyTask(*args, **kwargs)

Bases: fireworks.core.firework.FiretaskBase

Runs any python function! Extremely powerful, which allows you to essentially run any accessible method on the system. The optional inputs and outputs lists may contain spec keys to add to args list and to make the function output available in the current and in children fireworks.

Required parameters:
  • func (str): Fully qualified python method. E.g., json.dump, or shutil

    .copy, or some other function that is not part of the standard library!

Optional parameters:

  • args (list): List of args. Default is empty.

  • kwargs (dict): Dictionary of keyword args. Default is empty.

  • auto_kwargs (bool): If True, all other params not starting with ‘_’ are supplied as keyword args

  • stored_data_varname (str): Whether to store the output in FWAction. If this is a string that does not evaluate to False, the output of the function will be stored as FWAction(stored_data={stored_data_varname: output}). The name is deliberately long to avoid potential name conflicts.

  • inputs ([str]): a list of keys in spec which will be used as inputs; the generated arguments list will be appended to args

  • outputs ([str]): a list of spec keys that will be used to pass the function’s outputs to child fireworks

  • chunk_number (int): a serial number of the Firetask within a group of Firetasks generated by a ForeachTask

other_params = ['args', 'kwargs', 'auto_kwargs', 'stored_data_varname', 'inputs', 'outputs', 'chunk_number']
required_params = ['func']
run_task(fw_spec: Dict[str, Union[List[int], int]]) → Optional[fireworks.core.firework.FWAction]

This method gets called when the Firetask is run. It can take in a Firework spec, perform some task using that data, and then return an output in the form of a FWAction.

Parameters

fw_spec (dict) – A Firework spec. This comes from the master spec. In addition, this spec contains a special “_fw_env” key that contains the env settings of the FWorker calling this method. This provides for abstracting out certain commands or settings. For example, “foo” may be named “foo1” in resource 1 and “foo2” in resource 2. The FWorker env can specify { “foo”: “foo1”}, which maps an abstract variable “foo” to the relevant “foo1” or “foo2”. You can then write a task that uses fw_spec[“_fw_env”][“foo”] that will work across all these multiple resources.

Returns

(FWAction)

class fireworks.user_objects.firetasks.script_task.ScriptTask(*args, **kwargs)

Bases: fireworks.core.firework.FiretaskBase

Runs a user-defined script

classmethod from_str(shell_cmd, parameters=None)
required_params = ['script']
run_task(fw_spec)

This method gets called when the Firetask is run. It can take in a Firework spec, perform some task using that data, and then return an output in the form of a FWAction.

Parameters

fw_spec (dict) – A Firework spec. This comes from the master spec. In addition, this spec contains a special “_fw_env” key that contains the env settings of the FWorker calling this method. This provides for abstracting out certain commands or settings. For example, “foo” may be named “foo1” in resource 1 and “foo2” in resource 2. The FWorker env can specify { “foo”: “foo1”}, which maps an abstract variable “foo” to the relevant “foo1” or “foo2”. You can then write a task that uses fw_spec[“_fw_env”][“foo”] that will work across all these multiple resources.

Returns

(FWAction)

fireworks.user_objects.firetasks.templatewriter_task module

This module contains the TemplateWriterTask, which writes files based on a template file and a Context using Jinja2’s templating engine.

class fireworks.user_objects.firetasks.templatewriter_task.TemplateWriterTask(*args, **kwargs)

Bases: fireworks.core.firework.FiretaskBase

Task to write templated files via Jinja2 library Required parameters:

  • template_file: (str) - path to template file

  • context: (dict) - variable replacements for the template file

  • output_file: (str) - output file

Optional parameters:

  • append: (bool) - append to output file (instead of overwrite)

  • template_dir: (str) - directory in which to find the template file

run_task(fw_spec)

This method gets called when the Firetask is run. It can take in a Firework spec, perform some task using that data, and then return an output in the form of a FWAction.

Parameters

fw_spec (dict) – A Firework spec. This comes from the master spec. In addition, this spec contains a special “_fw_env” key that contains the env settings of the FWorker calling this method. This provides for abstracting out certain commands or settings. For example, “foo” may be named “foo1” in resource 1 and “foo2” in resource 2. The FWorker env can specify { “foo”: “foo1”}, which maps an abstract variable “foo” to the relevant “foo1” or “foo2”. You can then write a task that uses fw_spec[“_fw_env”][“foo”] that will work across all these multiple resources.

Returns

(FWAction)

fireworks.user_objects.firetasks.unittest_tasks module

class fireworks.user_objects.firetasks.unittest_tasks.ExportTestSerializer(a)

Bases: fireworks.utilities.fw_serializers.FWSerializable

__init__(a)

Initialize self. See help(type(self)) for accurate signature.

classmethod from_dict(m_dict)
to_dict()
class fireworks.user_objects.firetasks.unittest_tasks.TestSerializer(a, m_date)

Bases: fireworks.utilities.fw_serializers.FWSerializable

__init__(a, m_date)

Initialize self. See help(type(self)) for accurate signature.

classmethod from_dict(m_dict)
to_dict(*args, **kwargs)

Module contents