Internet Archive Scholar

Workflow task clustering for best effort systems with Pegasus

Gurmeet Singh, Mei-Hui Su, Karan Vahi, Ewa Deelman, Bruce Berriman, John Good, Daniel S. Katz, Gaurang Mehta
2008 Proceedings of the 15th ACM Mardi Gras conference on From lightweight mash-ups to lambda grids: Understanding the spectrum of distributed computing requirements, applications, tools, infrastructures, interoperability, and the incremental adoption of key capabilities - MG '08  

Preserved Fulltext

fulltext thumbnail
Web Archive Capture PDF (152.9 kB)
https://web.archive.org/web/20080524213732/http://www.isi.edu/~gurmeet/papers/MardiGras_v3.pdf

A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2008; you can also visit the original URL. The file type is application/pdf.

Many scientific workflows are composed of fine computational granularity tasks, yet they are composed of thousands of them and are data intensive in nature, thus requiring resources such as the TeraGrid to execute efficiently. In order to improve the performance of such applications, we often employ task clustering techniques to increase the computational granularity of workflow tasks. The goal is to minimize the completion time of the workflow by reducing the impact of queue wait times. In
more » ... paper, we examine the performance impact of the clustering techniques using the Pegasus workflow management system. Experiments performed using an astronomy workflow on the NCSA TeraGrid cluster show that clustering can achieve a significant reduction in the workflow completion time (upto 97%).
doi:10.1145/1341811.1341822 dblp:conf/mg/SinghSVDBGKM08 fatcat:bte7n5pjxzgmdpsrugxb5wtllu
Citation

Gurmeet Singh, Mei-Hui Su, Karan Vahi, Ewa Deelman, Bruce Berriman, John Good, Daniel S. Katz, Gaurang Mehta. "Workflow task clustering for best effort systems with Pegasus." Proceedings of the 15th ACM Mardi Gras conference on From lightweight mash-ups to lambda grids: Understanding the spectrum of distributed computing requirements, applications, tools, infrastructures, interoperability, and the incremental adoption of key capabilities - MG '08 (2008)