12,534 Hits in 5.6 sec

Mechanisms for provenance collection in scientific workflow systems

Mehdi Sarikhani, Andrew Wendelborn
2017 Computing  
Scientists in different disciplines use scientific workflows as management and representational frameworks for distributed scientific computations.  ...  Scientific workflow systems need a scientific workflow management system (SWfMS) to manage the flow of work among (both local and distributed) participants and resources; and to coordinate user and system  ...  AN!ADAPTIVE!PROVENANCE!ARCHITECTURE!IN!SCIENTIFIC!WORKFLOW! ...................................! 169! 6.1! ADAPTIVE!PROVENANCE!IN!SCIENTIFIC!WORKFLOW!  ... 
doi:10.1007/s00607-017-0578-1 fatcat:wcva2wnyqjhtlh3dmoidbamdqu

Provenance Holder: Bringing Provenance, Reproducibility and Trust to Flexible Scientific Workflows and Choreographies [chapter]

Ludwig Stage, Dimka Karastoyanova
2019 Lecture Notes in Business Information Processing  
We contribute the architecture of the Provenance Holder, its components, functionality and two main operations: recording provenance data and retrieving the trusted provenance information.  ...  The main principles and techniques used are related to the adaptation of workflows and choreographies, as well as to systems integration and software architecture improvement.  ...  Requirements on Workflow Management Systems The Provenance Holder as introduced above requires that the WfMSs are able to sign, for example, the provenance data they produce the executed workflow version  ... 
doi:10.1007/978-3-030-37453-2_53 fatcat:ctchsa2iynfsfnucbf73fxc46y

Collaborative Scientific Workflows

Shiyong Lu, Jia Zhang
2009 2009 IEEE International Conference on Web Services  
In recent years, a number of scientific workflow management systems (SWFMSs) have been developed to help domain scientists synergistically integrate distributed computations, datasets, and analysis tools  ...  This paper reviews the state of the art of the field of scientific workflows towards the support of collaborative scientific workflows, identifies critical research challenges, and presents our ongoing  ...  Collaboration is separated from scientific workflow management into two layers. Provenance data models and management models are handled by a dedicated Data Architecture layer.  ... 
doi:10.1109/icws.2009.150 dblp:conf/icws/LuZ09 fatcat:uixs7xfwond7ncnairul5yheje

Collaborative scientific workflows supporting collaborative science

Shiyong Lu, Jia Zhang
2011 International Journal of Business Process Integration and Management  
Recently, scientific workflows have emerged for scientists to integrate distributed computations, datasets, and analysis tools to enable and accelerate scientific discovery.  ...  This paper presents a disciplinary definition of this term, discusses the opportunities, requirements, and challenges of collaborative scientific workflows for the enablement of scientific collaboration  ...  Collaboration is separated from scientific workflow management into two layers. Provenance data models and management models are handled by a dedicated Data Architecture layer.  ... 
doi:10.1504/ijbpim.2011.040209 fatcat:5op5on2xgffhjabh7wgjrfuhai

Towards Semantic Provenance in CRISTAL

Jetendr Shamdasani, Richard McClatchey, Andrew Branson
2012 Extended Semantic Web Conference  
Traceability is an important feature of workflow based systems, and is a key source of provenance data.  ...  In this paper we summarize some initial work towards the adaptation of CRISTAL to a more semantic orientation, in particular compliance with the Open Provenance Model.  ...  It is a distributed data and workflow management system which uses a database for its repository, a multi-layered architecture for its component abstraction and dynamic object modelling for the design  ... 
dblp:conf/esws/ShamdasaniMB12 fatcat:bqajaiuufnfrdkuutk6escsw6m

Towards Provenance and Traceability in CRISTAL for HEP

Jetendr Shamdasani, Andrew Branson, Richard McClatchey
2014 Journal of Physics, Conference Series  
This paper discusses the CRISTAL object lifecycle management system and its use in provenance data management and the traceability of system events.  ...  From these examples, applications are drawn for the High Energy Physics domain and some initial ideas for its use in data preservation HEP are outlined in detail in this paper.  ...  It is a distributed data and workflow management system which uses a database for its repository, a multi-layered architecture for its component abstraction and dynamic object modelling for the design  ... 
doi:10.1088/1742-6596/513/3/032091 fatcat:425xcbkhrzbolfycwxkx2t5xee

Opportunities and Challenges in Running Scientific Workflows on the Cloud

Yong Zhao, Xubo Fei, Ioan Raicu, Shiyong Lu
2011 2011 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery  
We coin the term "Cloud Workflow", to refer to the specification, execution, provenance tracking of large-scale scientific workflows, as well as the management of data and computing resources to enable  ...  Cloud computing; Scientific Workflow; Cloud workflow; Data Intensive Computing I.  ...  and distributed services and software tools integration; (R4) Heterogeneous and distributed data product management; (R5) High-end computing support; (R6) Workflow monitoring and failure handling; and  ... 
doi:10.1109/cyberc.2011.80 dblp:conf/cyberc/ZhaoFRL11 fatcat:zlpe7p73lnh2jksgtnstvkafn4


Michael Rovatsos, Dimitrios I. Diochnos, Zhenyu Wen, Sofia Ceppi, Pavlos Andreadis
2017 Proceedings of the Symposium on Applied Computing - SAC '17  
This is accomplished by learning how to propose and route human-based tasks, how to allocate computational resources when managing these tasks, and how to adapt the overall interaction model of the platform  ...  distributed computation.  ...  Diversity-Aware Collective Adaptive Systems: Where people meet machines to build smarter societies" (  ... 
doi:10.1145/3019612.3019623 dblp:conf/sac/RovatsosDWCA17 fatcat:ol7cfuzgazgupc5uird6wvhzla

Distributed in-memory data management for workflow executions

Renan Souza, Vitor Silva, Alexandre A. B. Lima, Daniel de Oliveira, Patrick Valduriez, Marta Mattoso
2021 PeerJ Computer Science  
In this work, we present SchalaDB, an architecture with a set of design principles and techniques based on distributed in-memory data management for efficient workflow execution control and user steering  ...  We propose a distributed data design for scalable workflow task scheduling and high availability driven by a parallel and distributed in-memory DBMS.  ...  The authors would also like to thank Pedro Paiva Miranda for his help during the development of d-Chiron.  ... 
doi:10.7717/peerj-cs.527 pmid:34013039 pmcid:PMC8114816 fatcat:lg2cmuwsgvdzzk3mowg7e5doea

Provenance for Scientific Workflows Towards Reproducible Research

Roger S. Barga, Yogesh L. Simmhan, Eran Chinthaka, Satya Sanket Sahoo, Jared Jackson, Nelson Araujo
2010 IEEE Data Engineering Bulletin  
Trident provides an integrated way to collect, store and view provenance for workflows, and supports a range of provenance queries over workflow results.  ...  In the following sections, we discuss the interoperable data model used to represent provenance in Trident, means for distributed collection of provenance from the various workbench components, and its  ... 
dblp:journals/debu/BargaSCSJA10 fatcat:vau5nyssgvc7xlkry2xylwr3qe

The Deployment of an Enhanced Model-Driven Architecture for Business Process Management [article]

Richard McClatchey
2018 arXiv   pre-print
to the underlying data schema and enabling the gathering of traceable (provenance) data.  ...  The CRISTAL software, which originated at CERN for handling physics data, uses versions of stored descriptions to define versions of data and workflows which can be evolved over time and thereby to handle  ...  ACKNOWLEDGEMENTS The author wishes to highlight the support of his home institute and acknowledges the support of the European Union for the CRISTAL-ISE project under the 2011-2012 Marie Curie Industry  ... 
arXiv:1803.07435v1 fatcat:k25gm2uzxrg25ptgpgviiirwhu

dtoolAI: Reproducibility for Deep Learning

Matthew Hartley, Tjelvar S.G. Olsson
2020 Patterns  
The FAIR principles for data stewardship and software/workflow implementation give excellent high-level guidance on ensuring effective reuse of data and software.  ...  The package implements automatic capture of provenance information during model training and simplifies model distribution.  ...  ACKNOWLEDGMENTS We would like to thank the Patterns editorial team and the reviewers of our manuscript for their constructive feedback that resulted in a muchimproved paper.  ... 
doi:10.1016/j.patter.2020.100073 pmid:33205122 pmcid:PMC7660391 fatcat:zh5ao6j22fefbplaj53jh55bji

Scalable Workflow-Driven Hydrologic Analysis in HydroFrame [chapter]

Shweta Purawat, Cathie Olschanowsky, Laura E. Condon, Reed Maxwell, Ilkay Altintas
2020 Lecture Notes in Computer Science  
Towards this goal, we present a design that leverages provenance data and machine learning techniques to predict performance and forecast failures using an automatic performance collection component of  ...  We demonstrate how different modules of the workflow can be reused and repurposed for the three target user groups.  ...  This work is supported by NSF OAC CSSI 1835855, and DOE DE-SC0012630 for IPPD.  ... 
doi:10.1007/978-3-030-50371-0_20 fatcat:c6632vj24nd25dcdq73i6bxhyu

Examining the Challenges of Scientific Workflows

Yolanda Gil, Ewa Deelman, Mark Ellisman, Thomas Fahringer, Geoffrey Fox, Dennis Gannon, Carole Goble, Miron Livny, Luc Moreau, Jim Myers
2007 Computer  
Workflows have recently emerged as a paradigm for representing and managing complex distributed scientific computations and therefore accelerate the pace of scientific progress.  ...  A recent workshop on the Challenges of Scientific Workflows, sponsored by the National Science Foundation and held on May 1-2, 2006, brought together domain scientists, computer scientists, and social  ...  We would like to thank Maria Zemankova, Program Manager of the Information and Intelligent Systems Division, for supporting the workshop and contributing to the discussions.  ... 
doi:10.1109/mc.2007.421 fatcat:2lhu5j6snngvxj4a5uu335v4ja

The future of scientific workflows

Ewa Deelman, Tom Peterka, Ilkay Altintas, Christopher D Carothers, Kerstin Kleese van Dam, Kenneth Moreland, Manish Parashar, Lavanya Ramakrishnan, Michela Taufer, Jeffrey Vetter
2017 The international journal of high performance computing applications  
of emerging extreme-scale computing systems on those workflows, and to develop requirements for automated workflow management in future and existing environments.  ...  tool for extreme-scale science.  ...  Acknowledgments The authors would like to thank all workshop participants for their contributions: Greg Abram, Gagan Agrawal, James  ... 
doi:10.1177/1094342017704893 fatcat:wkqlq7kzlnfx7acppift6ot4u4
« Previous Showing results 1 — 15 out of 12,534 results