Bringing Scientific Workflows to Amazon SWF

Matthias Janetschek, Simon Ostermann, Radu Prodan
2013 2013 39th Euromicro Conference on Software Engineering and Advanced Applications  
In response to the ever-increasing needs of scientific applications for resources, Cloud computing emerged as an alternative on-demand and cost-effective resource provisioning approach. In this context, Cloud providers have recognised the importance of workflow applications to science and provide their own native solutions, such as the Amazon Simple Workflow Service (SWF). Nevertheless, an important downside of SWF is its incompatibility with existing workflow systems, and lack of means for
more » ... ing scientific legacy code. Similarly, existing workflow middlewares and applications require non-trivial extensions to take advantage of Cloud resources. We present in this paper a software engineering solution that allows the scientific workflow community access the Amazon Cloud through one single frontend converter, and propose a legacy wrapper service for executing legacy code using SWF. Empirical results using a real-world scientific workflow demonstrate that our automatically generated SWF application performs almost as fast as a native manuallyoptimised version, and outperforms other workflow middleware systems using the Amazon Cloud. Listing 6. Data flow within the ParallelForEach_1 composite activity. 1 p r i v a t e Promise p a r a l l e l F o r E a c h 1 ( . . . ) { 2 i n t maxIter = PLMTars . g e t ( ) . l e n g t h ; 3 / / Holds o u t p u t v a l u e s o f l i n e a r M o d e l a c t i v i t y 4 Promise [ ] o u t 1 = new Promise [ maxIter ] ; 5 / / Holds o u t p u t v a l u e s o f d e c i s i o n N o d e 1 a c t i v i t y 6 Promise [ ] o u t 2 = new Promise [ maxIter ] ; 7 f o r ( i n t i = 0 ; i < maxIter ; i ++) { 8 Promise p = Promise . a s P r o m i s e ( PLMTars . g e t ( ) [ i ] ; 9 / / Save l i n e a r M o d e l r e t u r n v a l u e o f i n lmo1 10 Promise lmo1 = a c t i v i t y C l i e n t . l i n e a r M o d e l ( p ) ; 11 / / Convert lmo1 i n a format f o r f u r t h e r p r o c e s s i n g 12 Promise [ ] lmo2 = U t i l s . convertPWA2Pa ( lmo1 , 2 ) ; 13 / / I n p u t f i r s t v a l u e s t o r e d i n lmo2 ; s a v e r e t u r n v a l u e i n t o dno1 14 Promise dno1 = d e c i s i o n N o d e 1 ( lmo2 [ 0 ] , isPPS ) ; 15 / / S t o r e l i n e a r M o d e l r e t u r n v a l u e i n a c o l l e c t i o n 16 o u t 1 [ i ] = lmo2 [ 1 ] ; 17 / / S t o r e i f r e t u r n v a l u e i n a c o l l e c t i o n 18 o u t 2 [ i ] = dno1 ; 19 } 20 Promise [ ] r e t v a l = new Promise [ 2 ] ; 21 / / Convert c o l l e c t i o n t o a s u i t a b l e r e t u r n format 22
doi:10.1109/seaa.2013.13 dblp:conf/euromicro/Janetschek0P13 fatcat:cfo6xwcpcrdbphg2gvtuk2gmxa