Two-level Dynamic Workflow Orchestration in the INDIGO DataCloud for Large-scale, Climate Change Data Analytics Experiments

Marcin Płóciennik, Sandro Fiore, Giacinto Donvito, Michał Owsiak, Marco Fargetta, Roberto Barbera, Riccardo Bruno, Emidio Giorgio, Dean N. Williams, Giovanni Aloisio
2016 Procedia Computer Science  
In this paper we present the approach proposed by EU H2020 INDIGO-DataCloud project to orchestrate dynamic workflows over a cloud environment. The main focus of the project is on the development of open source Platform as a Service solutions targeted at scientific communities, deployable on multiple hardware platforms, and provisioned over hybrid e-Infrastructures. The project is addressing many challenging gaps in current cloud solutions, responding to specific requirements coming from
more » ... ic communities including Life Sciences, Physical Sciences and Astronomy, Social Sciences and Humanities, and Environmental Sciences. We are presenting the ongoing work on implementing the whole software chain on the Infrastructure as a Service, PaaS and Software as a Service layers, focusing on the scenarios involving scientific workflows and big data analytics frameworks. INDIGO module for Kepler worflow system has been introduced along with the INDIGO underlying services exploited by the workflow components. A climate change data analytics experiment use case regarding the precipitation trend analysis on CMIP5 data is described, that makes use of Kepler and big data analytics services.
doi:10.1016/j.procs.2016.05.359 fatcat:5pd4w5up5bhcvgt4bxns3h5jnu