A Testing Engine for High-Performance and Cost-Effective Workflow Execution in the Cloud

V.K. Pallipuram, T. Estrada, M. Taufer
2015 2015 44th International Conference on Parallel Processing  
While pursuing high performance and cost effectiveness for directed acyclic graph (DAG)-structured scientific workflow executions in the cloud, it is critical to identify appropriate resource instances and their quantity. This paper presents a testing engine that employs a resource-selection heuristic, which statically analyzes the DAG structure to guide the selection of resource instances, how many and which ones. The testing engine combines the heuristic with two platformindependent
more » ... ling policies, the Area-oriented DAGscheduling heuristic AO (AO) and the Locally-Optimal heuristic (L-OPT), to perform extensive validation assessments. The testing engine ensures the realism of these assessments by modeling the performance variability of the cloud platform using real traces. The testing engine also enables cost-effectiveness analysis that guides users to select a small set of instance candidates that provide performance-cost tradeoff. Our empirical results show that the pairing of the resource-selection heuristic with AO scheduling policy is a powerful method for cost-effective DAGstructured workflow execution in the cloud.
doi:10.1109/icpp.2015.94 dblp:conf/icpp/PallipuramET15 fatcat:dkzwakicvzagvpl2mewxzk365u