138 Hits in 4.4 sec

Performance Evaluation of the Karma Provenance Framework for Scientific Workflows [chapter]

Yogesh L. Simmhan, Beth Plale, Dennis Gannon, Suresh Marru
2006 Lecture Notes in Computer Science  
The Karma provenance framework provides a means to collect workflow, process, and data provenance from data-driven scientific workflows and is used in the Linked Environments for Atmospheric Discovery  ...  This article presents a performance analysis of the Karma service as compared against the contemporary PReServ provenance service.  ...  The authors would like to thank Paul Groth from the University of Southampton for helping us deploy the PReServ server, the members of the LEAD team for their support and feedback on our work, and Abhijit  ... 
doi:10.1007/11890850_23 fatcat:gawmanmqkze6zpig25fo3pddi4

A Noisy 10GB Provenance Database [chapter]

You-Wei Cheah, Beth Plale, Joey Kendall-Morwick, David Leake, Lavanya Ramakrishnan
2012 Lecture Notes in Business Information Processing  
Provenance of scientific data is a key piece of the metadata record for the data's ongoing discovery and reuse.  ...  We discuss the process of generating the provenance database, and show early results on the kinds of provenance analysis enabled by the large provenance.  ...  This work was supported by the Director, Office of Science, of the U.S. Department of Energy under Contract No. DE-AC02-05CH11231.  ... 
doi:10.1007/978-3-642-28115-0_35 fatcat:6noglbhxpjevhgnjsp7m6m6bdy

A Framework for Collecting Provenance in Data-Centric Scientific Workflows

Yogesh Simmhan, Beth Plale, Dennis Gannon
2006 2006 IEEE International Conference on Web Services (ICWS'06)  
The framework, based on a loosely-coupled publish-subscribe architecture for propagating provenance activities, satisfies the needs of detailed provenance collection while a performance evaluation of a  ...  The focus of our work is on provenance collection for these workflows, necessary to validate the workflow and to determine quality of generated data products.  ...  Our evaluation of the framework implementation also shows that provenance collection can be done with minimal performance overhead on the workflow, on the order of 1% of the execution time for 271 files  ... 
doi:10.1109/icws.2006.5 dblp:conf/icws/SimmhanPG06 fatcat:yzo2pfks4bchxfezkslwjdsw64

Storing, reasoning, and querying OPM-compliant scientific workflow provenance using relational databases

Chunhyeok Lim, Shiyong Lu, Artem Chebotko, Farshad Fotouhi
2011 Future generations computer systems  
Experiments are conducted to evaluate the performance of OPMProv in data mapping and provenance querying.  ...  Provenance, the metadata that records the derivation history of scientific results, is essential in scientific workflows to support the reproducibility of scientific discovery, result interpretation, and  ...  Acknowledgments The authors would like to thank Girish Subramanian from Indiana University for helping us set up the provenance database of Karma.  ... 
doi:10.1016/j.future.2010.10.013 fatcat:mdzm4lhc7fcmpir6xxvvnb4nvm

WORKEM: Representing and Emulating Distributed Scientific Workflow Execution State

Lavanya Ramakrishnan, Dennis Gannon, Beth Plale
2010 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing  
There is a need for a customizable, isolated and manageable testing container for design, evaluation and deployment of distributed workflows.  ...  We also detail the use of the framework in two specific case studies (a) design and testing of an orchestration system (b) generation of a provenance database.  ...  Karma has a modular architecture that supports multiple types of data sources for provenance data.  ... 
doi:10.1109/ccgrid.2010.89 dblp:conf/ccgrid/RamakrishnanGP10 fatcat:kt4lvf5fwzerfc2rb3dx5pfkea

Managing Provenance in iRODS [chapter]

Andrea Weise, Adil Hasan, Mark Hedges, Jens Jensen
2009 Lecture Notes in Computer Science  
In this paper, we describe the provenance needs of iRODS and we survey briefly current provenance and provenance enabled workflow systems.  ...  Provenance data does not only give a history of events, it also provides enough information to allow the opportunity to verify the authenticity of the data, as well as, determine the quality of the data  ...  Acknowledgement This work, which is part of the ASPiS project, was funded by the UK Joint Information Systems Committee (JISC) as part of its e-Infrastructure programme, with additional support from UK  ... 
doi:10.1007/978-3-642-01973-9_75 fatcat:eg2jdbzqxvcrjnr67nyg23rrqi

Provenance for Scientific Workflows Towards Reproducible Research

Roger S. Barga, Yogesh L. Simmhan, Eran Chinthaka, Satya Sanket Sahoo, Jared Jackson, Nelson Araujo
2010 IEEE Data Engineering Bulletin  
Karma [11] uses instrumentation of workflow engine and Axis2 web service activities for recording provenance in a database, also using a pub-sub model for event transfer.  ...  These workflows may run on the users desktop or in the Cloud and the workflow framework is geared towards easy composition of scientific experiments, allocation and scheduling of resources, orchestration  ... 
dblp:journals/debu/BargaSCSJA10 fatcat:vau5nyssgvc7xlkry2xylwr3qe

Big Provenance Stream Processing for Data Intensive Computations

Isuru Suriarachchi, Sachith Withana, Beth Plale
2018 2018 IEEE 14th International Conference on e-Science (e-Science)  
Karma [88] is one of the first attempts to build a general purpose provenance collection framework for scientific workflows which uses a standard provenance representation language.  ...  It provides derivation traces for scientific data. MyGrid [111] is a provenance enabled middleware framework for SOA based workflows.  ...  Senior Technical Lead WSO2, Sri Lanka May 2008 -July 2012 • Worked as a core developer of the WSO2 Carbon framework, the OSGi based middleware platform for the entire WSO2 product stack. • Lead developer  ... 
doi:10.1109/escience.2018.00039 dblp:conf/eScience/SuriarachchiWP18 fatcat:xrkgn66wkzhyxm2z6duattyqdy

Using Domain-Specific Data to Enhance Scientific Workflow Steering Queries [chapter]

João Carlos de A.R. Gonçalves, Daniel de Oliveira, Kary A. C. S. Ocaña, Eduardo Ogasawara, Marta Mattoso
2012 Lecture Notes in Computer Science  
In scientific workflows, provenance data helps scientists in understanding, evaluating and reproducing their results.  ...  We have evaluated the proposed approach using a real bioinformatics workflow for comparative genomics executed in SciCumulus cloud workflow parallel engine.  ...  In the first category, the most similar approach to the one proposed in this paper is Karma [29] , a framework for collecting provenance from heterogeneous workflow environments.  ... 
doi:10.1007/978-3-642-34222-6_12 fatcat:f2lp4foydzddvo6itwspnbuygu

Report on the International Provenance and Annotation Workshop

Rajendra Bose, Ian Foster, Luc Moreau
2006 SIGMOD record  
Acknowledgment and thanks are due to the IPAW'06 program committee, and the hosts and meeting coordinators at the University of Chicago and Argonne National Laboratory.  ...  Yogesh Simmhan (Indiana University) presented a quantitative performance comparison of two methods of recording provenance for scientific workflow execution: the Karma framework and the Provenance Recording  ...  Ilkay Altintas (San Diego Supercomputing Center/University of California, San Diego) discussed a provenance framework for the open source-based Kepler scientific workflow system; this framework includes  ... 
doi:10.1145/1168092.1168102 fatcat:l6cgygk2bfc6zdq5ymoo2rvjce


Runzhou Han, Suren Byna, Houjun Tang, Bin Dong, Mai Zheng
2022 Proceedings of the 31st International Symposium on High-Performance Parallel and Distributed Computing  
Our experiments with realistic workflows show that PROV-IO can address the provenance needs of the domain scientists effectively with reasonable performance (e.g., less than 3.5% tracking overhead for  ...  Based on the first-hand analysis, we propose a provenance framework called PROV-IO, which includes an I/O-centric provenance model for describing scientific data and the associated I/O operations and environments  ...  The authors would like to thank Yogesh Simmhan (our shepherd) and the anonymous reviewers for their insightful feedback. We also thank Xiangyang Ju for providing the Top Reco workflow.  ... 
doi:10.1145/3502181.3531477 fatcat:3zf37byleff55ied7qkjwza5em

A survey of simulation provenance systems: modeling, capturing, querying, visualization, and advanced utilization

Young-Kyoon Suh, Ki Yong Lee
2018 Human-Centric Computing and Information Sciences  
In particular, we present a taxonomy of scientific platforms regarding provenance support and holistically tabulate the major functionalities and supporting levels of the studied systems.  ...  In this manuscript we provide a comprehensive survey of a wide range of existing systems to utilize provenance data produced by simulation.  ...  Acknowledgements Competing interests The authors declare that they have no competing interests.  ... 
doi:10.1186/s13673-018-0150-9 fatcat:zmdmunmguvfelnlwcvnhw5wmpi

Automated Provenance Collection for CCA Component Assemblies [chapter]

Kostadin Damevski, Hui Chen
2009 Lecture Notes in Computer Science  
The problem of capturing provenance for computational tasks has recently received significant attention, due to the new set of beneficial uses (for optimization, debugging, etc.) of the recorded data.  ...  We develop a provenance collection system aimed at scientific applications that are based on the Common Component Architecture (CCA) that alleviates scientists from the responsibility to manually instrument  ...  The Karma framework [11] collects provenance in the context of web service scientific applications.  ... 
doi:10.1007/978-3-642-01970-8_26 fatcat:33zb5wax5nfnpjfjeoamuk5uiu

Provenance Support for Grid-Enabled Scientific Workflows

Fakhri Alam Khan, Yuzhang Han, Sabri Pllana, Peter Brezany
2008 2008 Fourth International Conference on Semantics, Knowledge and Grid  
This issue is being addressed in our research reported in this paper via development of workflow provenance system for Grid-enabled scientific workflows.  ...  However, the credibility of the obtained results in the scientific community is questionable if the computational experiment is not reproducible.  ...  ACKNOWLEDGMENT This research is done in the context of the GridMiner [26] , [27] and ADMIRE [28] projects.  ... 
doi:10.1109/skg.2008.86 dblp:conf/skg/KhanHPB08 fatcat:5uunb2dskjchnonxqcgadt4q4e

Provenance analysis: Towards quality provenance

You-Wei Cheah, Beth Plale
2012 2012 IEEE 8th International Conference on E-Science  
We also establish crucial quality dimensions that are especially critical for the evaluation of provenance quality.  ...  Data provenance, a key piece of metadata that describes the lifecycle of a data product, is crucial in aiding scientists to better understand and facilitate reproducibility and reuse of scientific results  ...  We also thank Lavanya Ramakrishnan and Elif Dede for suggestions towards the writing of this paper.  ... 
doi:10.1109/escience.2012.6404480 dblp:conf/eScience/CheahP12 fatcat:et5ezqodibalrh7r5ljljgql5i
« Previous Showing results 1 — 15 out of 138 results