Heterogeneous Hierarchical Workflow Composition

Orcun Yildiz, Jorge Ejarque, Henry Chan, Subramanian Sankaranarayanan, Rosa M. Badia, Tom Peterka
2019 Computing in science & engineering (Print)  
Workflow systems promise scientists an automated end-to-end path from hypothesis to discovery. However, expecting any single workflow system to deliver such a wide range of capabilities is impractical. A more practical solution is to compose the end-to-end workflow from more than one system. With this goal in mind, the integration of task-based and in situ workflows is explored, where the result is a hierarchical heterogeneous workflow composed of subworkflows, with different levels of the
more » ... levels of the hierarchy using different programming, execution, and data models. Materials science use cases demonstrate the advantages of such heterogeneous hierarchical workflow composition. Scientific computing consists of multiple related computational tasks. For instance, the detection of highly turbulent potentially destructive features in plasma physics simulations requires the in situ coupling of two simulation codes, a compression tool, a feature detection algorithm, and a visualization library. 1 Such complex workflows require significant effort from scientists to manage the scheduling and data exchange among those tasks. To help automate this process, scientific workflow frameworks allow scientists to define the dependencies and data exchanges among connected tasks instead of managing those manually, potentially resulting in increased scientific productivity.
doi:10.1109/mcse.2019.2918766 fatcat:frwdvwfgxrhzrbzl5q6vxl2aw4