5 Hits in 1.4 sec

dispel4py: An Agile Framework for Data-Intensive eScience

Rosa Filgueira, Amrey Krause, Malcolm Atkinson, Iraklis Klampanos, Alessandro Spinuso, Susana Sanchez-Exposito
2015 2015 IEEE 11th International Conference on e-Science  
We present dispel4py a versatile data-intensive kit presented as a standard Python library. It empowers scientists to experiment and test ideas using their familiar rapid-prototyping environment.  ...  It delivers mappings to diverse computing infrastructures, including cloud technologies, HPC architectures and specialised data-intensive machines, to move seamlessly into production with large-scale data  ...  CONCLUSIONS AND FUTURE WORK In this paper we presented dispel4py, a novel Python library for streaming, data-intensive processing.  ... 
doi:10.1109/escience.2015.40 dblp:conf/eScience/FilgueiraKAKSS15 fatcat:b5j3qwe2pzbrhlx2tlnbia7vsq

Comprehensible Control for Researchers and Developers Facing Data Challenges

Malcolm Atkinson, Rosa Filgueira, Iraklis Klampanos, Antonis Koukourikos, Amrey Krause, Federica Magnoni, Christian Page, Andreas Rietbrock, Alessandro Spinuso
2019 2019 15th International Conference on eScience (eScience)  
The DARE platform enables researchers and their developers to exploit more capabilities to handle complexity and scale in data, computation and collaboration.  ...  Today's challenges pose increasing and urgent demands for this combination of capabilities.  ...  The authors also thank Ghita Berrada, Oscar Corcho and Luca Trani for advice on early drafts and the developers and systems teams that have delivered and tested the platform.  ... 
doi:10.1109/escience.2019.00042 dblp:conf/eScience/AtkinsonFKKKMPR19 fatcat:o56yf66b4rcqxb3gwq7xmyshee

Active Provenance for Data-Intensive Workflows: Engaging Users and Developers

Alessandro Spinuso, Malcolm Atkinson, Federica Magnoni
2019 2019 15th International Conference on eScience (eScience)  
We present a practical approach for provenance capturing in Data-Intensive workflow systems. It provides contextualisation by recording injected domain metadata with the provenance stream.  ...  Provenance Configuration, instead, enables users of a Data-Intensive workflow execution o prepare it for provenance capture, by configuring the attribution of Provenance Types to components and by specifying  ...  The framework is available for data-streaming applications using dispel4py [19] mapped to several platforms.  ... 
doi:10.1109/escience.2019.00077 dblp:conf/eScience/SpinusoAM19 fatcat:omeuylderba3vhqsswwahqve3q

D2.1 Dare Architecture and Technical Positioning

Malcolm Atkinson, Emanuele Casarotti, Malin Ewering, Rosa Filgueira, André Gemünd, Iraklis Klampanos, Antonis Koukourikos, Amrey Krause, Federica Magnoni, Andrea Pagani, Christian Pagé, Andreas Rietbrock (+2 others)
2018 Zenodo  
The requirements for large-scale data-driven research with high data volume, computational and complexity.  ...  This includes a knowledge base, a workflows-as-a- sevice and a protected, persistent, pervasive provenance system.  ...  DARE will demonstrate how these can be brought into the remit of a DARE platform and used effectively to meet the application domains' challenges. dispel4py: A Python framework for describing abstract  ... 
doi:10.5281/zenodo.2613550 fatcat:s32p3orrerev5einzlmils34mu

D3.3 and D4.4: Consolidated Science Demonstrator progress and evaluation reports [article]

Hermann Lederer, Steven Newhouse
2018 Zenodo  
To achieve this goal, a selection process for Science Demonstrators through Open Calls has been developed, followed by the execution of two Open Calls.  ...  Feedback was collected in reports which are available for all work packages.  ...  Their data handling is made flexible and scalable by communitydeveloped libraries, such as ObsPy (, and data-intensive tools, such as dispel4py (  ... 
doi:10.5281/zenodo.3387596 fatcat:7svhwma7xrhyfch4qiqeaquruu