Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

To facilitate evaluation of workflow algorithms and systems on a range of workflow sizes, we have developed a set of synthetic workflow generator. This generator uses generators. These generators use the information gathered from actual executions of scientific workflows on the Grid as well as our understanding of the processes behind these workflows to generate realistic, synthetic workflows resembling those used by real world scientific applications.

Code

The code used to generate all of the workflows below, and many others, is available from the GitHub repository. The java workflow generator sometimes generates negative task runtimes, so watch out for that.

Pegasus Workflows

These workflows come from a paper by Bharathi, et al. [1]. There is another paper with more information about the workflows by Juve, et al. [2].

The code used to generate these workflows is available here. The code generator sometimes generates negative task runtimes, so watch out for that.

A large collection of DAXes similar to the ones listed below is available here. Note that it is about 375 MB.

...

These workflows come from a report by Ramakrishnan and Gannon [3].The Python code used to generate the DAX files below, as well as several others, can be downloaded here.

Workflow TypeFigure in ReportExampleDAX
LEAD Mesoscale MeteorologyFigure 1leadmm.xml
LEAD ARPS Data Analysis SystemFigure 2

leadadas.xml

LEAD Data Mining WorkflowFigure 3leaddm.xml
Storm Surge SCOOP WorkflowFigure 4

scoop_small.xml

scoop_medium.xml

scoop_large.xml

Floodplain MappingFigure 5floodplain.xml
GlimmerFigure 6glimmer.xml
Gene2LifeFigure 7gene2life.xml
Motif NetworkFigure 8

motif_small.xml

motif_medium.xml

motif_large.xml

MEME-MASTFigure 9mememast.xml
Molecular SciencesFigure 10molsci.xml
Avian FluFigure 11

avianflu_small.xml

avianflu_medium.xml

avianflu_large.xml

caDSRFigure 12cadsr.xml
Pan-STARRS LoadFigure 13

psload_small.xml

psload_medium.xml

psload_large.xml

Pan-STARRS MergeFigure 14

psmerge_small.xml

psmerge_medium.xml

psmerge_large.xml

McStasFigure 15mcstas.xml

...

 

[1] R. F. da Silva, W. Chen, G. Juve, K. Vahi, E. Deelman. Community Resources for Enabling Research in Distributed Scientific Workflows. 10th IEEE International Conference on e-Science (eScience 2014)

...