A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2018; you can also visit the original URL.
The file type is
We present dispel4py a versatile data-intensive kit presented as a standard Python library. It empowers scientists to experiment and test ideas using their familiar rapid-prototyping environment. ... It delivers mappings to diverse computing infrastructures, including cloud technologies, HPC architectures and specialised data-intensive machines, to move seamlessly into production with large-scale data ... CONCLUSIONS AND FUTURE WORK In this paper we presented dispel4py, a novel Python library for streaming, data-intensive processing. ...doi:10.1109/escience.2015.40 dblp:conf/eScience/FilgueiraKAKSS15 fatcat:b5j3qwe2pzbrhlx2tlnbia7vsq
The DARE platform enables researchers and their developers to exploit more capabilities to handle complexity and scale in data, computation and collaboration. ... Today's challenges pose increasing and urgent demands for this combination of capabilities. ... The authors also thank Ghita Berrada, Oscar Corcho and Luca Trani for advice on early drafts and the developers and systems teams that have delivered and tested the platform. ...doi:10.1109/escience.2019.00042 dblp:conf/eScience/AtkinsonFKKKMPR19 fatcat:o56yf66b4rcqxb3gwq7xmyshee
We present a practical approach for provenance capturing in Data-Intensive workflow systems. It provides contextualisation by recording injected domain metadata with the provenance stream. ... Provenance Configuration, instead, enables users of a Data-Intensive workflow execution o prepare it for provenance capture, by configuring the attribution of Provenance Types to components and by specifying ... The framework is available for data-streaming applications using dispel4py  mapped to several platforms. ...doi:10.1109/escience.2019.00077 dblp:conf/eScience/SpinusoAM19 fatcat:omeuylderba3vhqsswwahqve3q
The requirements for large-scale data-driven research with high data volume, computational and complexity. ... This includes a knowledge base, a workflows-as-a- sevice and a protected, persistent, pervasive provenance system. ... DARE will demonstrate how these can be brought into the remit of a DARE platform and used effectively to meet the application domains' challenges. dispel4py: A Python framework for describing abstract ...doi:10.5281/zenodo.2613550 fatcat:s32p3orrerev5einzlmils34mu
To achieve this goal, a selection process for Science Demonstrators through Open Calls has been developed, followed by the execution of two Open Calls. ... Feedback was collected in reports which are available for all work packages. ... Their data handling is made flexible and scalable by communitydeveloped libraries, such as ObsPy (http://obspy.org), and data-intensive tools, such as dispel4py (https://github.com/dispel4py/dispel4py) ...doi:10.5281/zenodo.3387596 fatcat:7svhwma7xrhyfch4qiqeaquruu