3,241 Hits in 9.0 sec

Towards A Methodology and Framework for Workflow-Driven Team Science [article]

Ilkay Altintas, Shweta Purawat, Daniel Crawl, Alok Singh, Kyle Marcus
2019 arXiv   pre-print
However, data and computing advances continuously change the way scientific workflows get developed and executed, pushing the scientific activity to be more data-driven, heterogeneous and collaborative  ...  Scientific workflows are powerful tools for management of scalable experiments, often composed of complex tasks running on distributed resources.  ...  The authors would also like to thank Workflows for Data Science (WorDS) Center of Excellence team members Mai Nguyen and Volkan Vural for their participation in the discussions leading to this paper.  ... 
arXiv:1903.01403v1 fatcat:ubbe6qk5kfb77erhetvp25jele

DARE: A Reflective Platform Designed to Enable Agile Data-Driven Research on the Cloud

Iraklis Klampanos, Federica Magnoni, Emanuele Casarotti, Christian Page, Mike Lindner, Andreas Ikonomopoulos, Vangelis Karkaletsis, Athanasios Davvetas, Andre Gemund, Malcolm Atkinson, Antonios Koukourikos, Rosa Filgueira (+3 others)
2019 2019 15th International Conference on eScience (eScience)  
It implements the cataloguing and execution of fine-grained and Python-based dispel4py workflows as services.  ...  The platform is Cloud-ready, and relies on the exposure of API, which are suitable for raising the abstraction level and hiding complexity.  ...  Index Terms-software platform, cloud, technology, conceptualization, data-driven science, scientific workflows, provenance, workflow optimization I.  ... 
doi:10.1109/escience.2019.00079 dblp:conf/eScience/KlampanosMCPLIK19 fatcat:7immlbim2racxa534ovdylysfi

Moriarty: Improving 'Time To Market' in big data and Artificial intelligence applications

P. Peña, R. Del Hoyo, J. Vea-murguía, V. Rodrigálvarez, J.I. Calvo, J.M. Martín
2016 International Journal of Design & Nature and Ecodynamics  
The data scientist works with algorithms and data transformations using a visual interface, while the software engineer works with the idea of services to be invoked.  ...  Moriarty is divided into several layers; its core is a BPMN engine, which executes the processing and defines data analytics process, called workflows.  ...  ACKNOWLEDGEMENTS The dissemination of this work has been partly financed by the FSE Operative Programme for Aragon (2014-2020).  ... 
doi:10.2495/dne-v11-n3-230-238 fatcat:6tidnqrkwnaj7fumhnme7dqqyq

Introducing a New Scalable Data-as-a-Service Cloud Platform for Enriching Traditional Text Mining Techniques by Integrating Ontology Modelling and Natural Language Processing [chapter]

Alexey Cheptsov, Axel Tenschert, Paul Schmidt, Birte Glimm, Mauricio Matthesius, Thorsten Liebig
2014 Lecture Notes in Computer Science  
The major challenge is the size, structural complexity, and frequency of the analysed text sets' updates (i.e., the 'big data' aspect), which makes the use of traditional analysis techniques and tools  ...  to obtain some useful, often not explicitly stated knowledge and facts, related to a particular domain of interest.  ...  software stack, such as execution runtime, database, web services host, workflow manager, etc., are permanently deployed on the cloud.  ... 
doi:10.1007/978-3-642-54370-8_6 fatcat:qoxc6xntnbczlp247iak3fb7dm

UBioLab: a web-LABoratory for Ubiquitous in-silico experiments

E. Bartocci, M. R. Di Berardini, E. Merelli, L. Vito
2012 Journal of Integrative Bioinformatics  
SummaryThe huge and dynamic amount of bioinformatic resources (e.g., data and tools) available nowadays in Internet represents a big challenge for biologists -for what concerns their management and visualization  ...  Several architectural features -as those ones of being fully Web-based and of combining domain ontologies, Semantic Web and workflow techniques- give evidence of an effort in such a direction.The integration  ...  A special thank to the students Victor Karmansky, Nicola Paoletti and Fabio Alessandrelli that gave an important contribution in the development of UBioLab.  ... 
doi:10.1515/jib-2012-192 fatcat:5ve4n2zxt5fwdf45v7kbpcfzse

UBioLab: a web-laboratory for ubiquitous in-silico experiments

Ezio Bartocci, Diletta Cacciagrano, Maria Rita Di Berardini, Emanuela Merelli, Leonardo Vito
2012 Journal of Integrative Bioinformatics  
The huge and dynamic amount of bioinformatic resources (e.g., data and tools) available nowadays in Internet represents a big challenge for biologists –for what concerns their management and visualization  ...  Several architectural features –as those ones of being fully Web-based and of combining domain ontologies, Semantic Web and workflow techniques– give evidence of an effort in such a direction.  ...  A special thank to the students Victor Karmansky, Nicola Paoletti and Fabio Alessandrelli that gave an important contribution in the development of UBioLab.  ... 
doi:10.2390/biecoll-jib-2012-192 pmid:22773116 fatcat:lipwrizxwnfclj4byghjlmjdta

Middleware Building Blocks for Workflow Systems [article]

Matteo Turilli, Vivek Balasubramanian, Andre Merzky, Ioannis Paraskevakos, Shantenu Jha
2019 arXiv   pre-print
, as well as providing a level of "unification" in the conceptual reasoning across otherwise very different tools and systems; and (iii) showing how these building blocks have been used to develop and  ...  Four case studies are presented, discussing how RADICAL-Cybertools are integrated with existing workflow, workload, and general purpose computing systems to support the execution of scientific workflows  ...  We thank Daniel Smith, Levi Naden and Sam Ellis (MolSSI) for useful discussions and insight. This was was supported primarily by NSF 1440677 and DOE ASCR de-sc0016280.  ... 
arXiv:1903.10057v1 fatcat:vvlkcxkuebhdxcp5bcqrcwu4tu

D4.1 Definition of Architecture for Extreme-Scale Analytics

Project Consortium Members
2019 Zenodo  
physical resources in a way that optimizes specific performance measures, (iii) providing real-time, interactive machine learning and data mining tools that can be leveraged by the designed workflows,  ...  by providing graphical workflow design facilities, (ii) automating the tuning of the underlying Big Data platform infrastructure that materializes the visually designed workflow as well as the provisioned  ...  analysis workflows to available compute cluster or a compute cluster of choice is a task which often requires to communicate with a specific Big Data platform using its custom APIs or client utilities  ... 
doi:10.5281/zenodo.4034092 fatcat:g766jj6xwvesddsm3xs56l6mqq

Web technologies for environmental Big Data

Claudia Vitolo, Yehia Elkhatib, Dominik Reusser, Christopher J.A. Macleod, Wouter Buytaert
2015 Environmental Modelling & Software  
the design of analysis methods, workflows, and interaction with data sets.  ...  We found that, the processing of the simple datasets used in the pilot proved to be relatively straightforward using a combination of R, RPy2, PyWPS and PostgreSQL.  ...  Acknowledgements This work was supported by the Natural Environment Research Council pilot projects on Environmental Virtual Observatory technologies NE/I002200/1 and NE/I004017/1.  ... 
doi:10.1016/j.envsoft.2014.10.007 fatcat:df3mpg7yunaehbfhzysbutze5y

A demonstration of modularity, reuse, reproducibility, portability and scalability for modeling and simulation of cardiac electrophysiology using Kepler Workflows

Pei-Chi Yang, Shweta Purawat, Pek U. Ieong, Mao-Tsuen Jeng, Kevin R. DeMarco, Igor Vorobyov, Andrew D. McCulloch, Ilkay Altinas, Rommie E. Amaro, Colleen E. Clancy, Herbert Sauro
2019 PLoS Computational Biology  
It invariably involves specific and detailed sequences of data analysis and simulation, often with multiple tools and datasets, and the community recognizes improved modularity, reuse, reproducibility,  ...  While there are good examples if the use of scientific workflows in bioinformatics, medical informatics, biomedical imaging and data analysis, there are fewer examples in multi-scale computational modeling  ...  Please access all codes and associated files and attributes via the GitHub link below. The repository contains specific instructions for use of Kepler System with new source codes.  ... 
doi:10.1371/journal.pcbi.1006856 pmid:30849072 pmcid:PMC6426265 fatcat:3gucgod6mfbsrf644otegioozm

Skyport - Container-Based Execution Environment Management for Multi-cloud Scientific Workflows

Wolfgang Gerlach, Wei Tang, Kevin Keegan, Travis Harrison, Andreas Wilke, Jared Bischof, Mark DSouza, Scott Devoid, Daniel Murphy-Olson, Narayan Desai, Folker Meyer
2014 2014 5th International Workshop on Data-Intensive Computing in the Clouds  
The portability and ease of deployment makes Linux containers an ideal technology to be used in scientific workflow platforms.  ...  As an extension to AWE/Shock, our data analysis platform that provides scalable workflow execution environments for scientific data in the cloud, Skyport greatly reduces the complexity associated with  ...  The use of shared file systems to distribute data for processing between machines traditionally has been a common strategy in many compute clusters but is less suited for big data applications.  ... 
doi:10.1109/datacloud.2014.6 dblp:conf/sc/GerlachTKHWBDDM14 fatcat:izi4g67uknh7hbemoyilt4cx7a

PhenoMeNal: processing and analysis of metabolomics data in the cloud

Kristian Peters, James Bradbury, Sven Bergmann, Marco Capuccini, Marta Cascante, Pedro de Atauri, Timothy M D Ebbels, Carles Foguet, Robert Glen, Alejandra Gonzalez-Beltran, Ulrich L Günther, Evangelos Handakas (+44 others)
2018 GigaScience  
However, the rapid progress has resulted in a mosaic of independent, and sometimes incompatible, analysis methods that are difficult to connect into a useful and complete data analysis solution.  ...  The research field is dynamic and expanding with applications across biomedical, biotechnological, and many other applied biological domains.  ...  Language CWL) Domain specific standards for raw data (e.g., mzML, nmrML) Containerization of software tools Execution in various software environments (e.g., through the use of containers) OboFoundry  ... 
doi:10.1093/gigascience/giy149 pmid:30535405 pmcid:PMC6377398 fatcat:srtrsdrzd5bitjztymyqy4nypy

Big Data Workflows: Locality-Aware Orchestration Using Software Containers

Andrei-Alin Corodescu, Nikolay Nikolov, Akif Quddus Khan, Ahmet Soylu, Mihhail Matskin, Amir H. Payberah, Dumitru Roman
2021 Sensors  
This article proposes a novel architecture and a proof-of-concept implementation for software container-centric big data workflow orchestration that puts data locality at the forefront.  ...  The proposed solution considers the available data locality information, leverages long-lived containers to execute workflow steps, and handles the interaction with different data sources through containers  ...  Conceptualization and scalable execution of big data workflows using domain-specific languages and software containers. Internet Things 2021, 100440, in press. [CrossRef] 13.  ... 
doi:10.3390/s21248212 pmid:34960302 pmcid:PMC8706844 fatcat:3nc2j4pvdfdynn573zzq7ympca

Toward Common Components for Open Workflow Systems [article]

Jay Jay Billings, Shantenu Jha
2017 arXiv   pre-print
The role of scalable high-performance workflows and flexible workflow management systems that can support multiple simulations will continue to increase in importance.  ...  This inconsistency often centers around the difference between several different types of workflows, including modeling and simulation, grid, uncertainty quantification, and purely conceptual workflows  ...  Billings would also like to acknowledge the thoughtful discussions with Jim Belak on the nature of workflows in the ExAM project, and Robert Clay, Dan Laney, and David Montoya on modeling and simulation  ... 
arXiv:1710.06774v1 fatcat:va4lwuennzabxhvbmii2muapga

A System Architecture for Running Big Data Workflows in the Cloud

Andrey Kashlev, Shiyong Lu
2014 2014 IEEE International Conference on Services Computing  
Finally, to validate our proposed architecture we conduct a case study in which we design and run a big data workflow towards addressing EB-scale big data analysis problem in the automotive industry domain  ...  Scientific workflows have become an important paradigm for domain scientists to formalize and structure complex data-intensive scientific processes.  ...  A scientific workflow management system (SWFMS) is a software system that allows domain scientists to design, store and execute scientific workflows to solve their domain problems.  ... 
doi:10.1109/scc.2014.16 dblp:conf/IEEEscc/KashlevL14 fatcat:4mmgfb7lafdfrovnpmvtgp7gfi
« Previous Showing results 1 — 15 out of 3,241 results