Automate Scientific Workflow Execution between Local Cluster and Cloud

Hao Qian, Daniel Andresen
2016 International Journal of Networked and Distributed Computing (IJNDC)  
Scientific computational experiments often span multiple computational and analytical steps, and during execution, researchers need to store, access, transfer, and query information. Scientific workflow is a powerful tool to streamline and organize scientific application. Numbers of tools have been developed to help build scientific workflows, they provide mechanisms for creating workflow but lack a native scheduling system for determining where code should be executed. This paper presents
more » ... ld, a system that adds sophisticated computation offloading capabilities to scientific workflows. Emerald automatically offloads computation intensive steps of scientific workflow to the cloud in order to enhance workflow performance. Emerald minimizes the burden on developers to build workflows with computation offloading ability by providing easy-to-use API. Evaluation showed that Emerald can effectively reduce up to 55% of execution time for scientific applications.
doi:10.2991/ijndc.2016.4.1.5 fatcat:j6bg3thp6rbvzpfle4tvnqkg5u