Filters








2 Hits in 1.8 sec

kiwiPy: Robust, high-volume, messaging for big-data and computational science workflows [article]

Martin Uhrin, Sebastiaan P. Huber
2020 arXiv   pre-print
In this work we present kiwiPy, a Python library designed to support robust message based communication for high-throughput, big-data, applications while being general enough to be useful wherever high-volumes  ...  KiwiPy relies on the RabbitMQ protocol, an industry standard message broker, while providing a simple and intuitive interface that can be used in both multithreaded and coroutine based applications.  ...  SUMMARY The computational sciences have seen a huge increase in the use of high-throughput, automated, workflows over the course of the last two decades or so.  ... 
arXiv:2005.07475v1 fatcat:hbtx3lw2czcfborzo7vyn4idtq

kiwiPy: Robust, high-volume, messaging for big-data and computational science workflows

Martin Uhrin, Sebastiaan Huber
2020 Journal of Open Source Software  
This work is supported by the MARVEL National Centre for Competency in Research funded by the Swiss National Science Foundation (grant agreement ID 51NF40-182892) and the European Materials Modelling Council-CSA  ...  Acknowledgements We would like to thank Giovanni Pizzi, Nicola Marzari, and the AiiDA team for their continuous coordination and development of the project.  ...  Uhrin et al.,(2020). kiwiPy: Robust, high-volume, messaging for big-data and computational science workflows. Journal of Open Source Software, 5(52), 2351. https://doi.org/10.21105/joss.02351  ... 
doi:10.21105/joss.02351 fatcat:dwsy7ivvmza7pewqwt54a4zlc4