6,435 Hits in 4.7 sec

Using Explicit Control Processes in Distributed Workflows to Gather Provenance [chapter]

Sérgio Manuel Serra da Cruz, Fernando Seabra Chirigati, Rafael Dahis, Maria Luiza M. Campos, Marta Mattoso
2008 Lecture Notes in Computer Science  
In these heterogeneous distributed environments, provenance gathering becomes also heterogeneous. This work presents control-flow modules that aim to be independent from WfMS.  ...  In addition, they can be used to gather provenance data both from local and remote execution, thus allowing the same provenance registration on both environments independent of the heterogeneous WfMS.  ...  In addition to reuse, we aim at gathering remote process provenance with the help of these control structures.  ... 
doi:10.1007/978-3-540-89965-5_20 fatcat:ro7547t2vvev5e3pj7legdiz4m

Exploring many task computing in scientific workflows

Eduardo Ogasawara, Daniel de Oliveira, Fernando Chirigati, Carlos Eduardo Barbosa, Renato Elias, Vanessa Braganholo, Alvaro Coutinho, Marta Mattoso
2009 Proceedings of the 2nd Workshop on Many-Task Computing on Grids and Supercomputers - MTAGS '09  
In addition, these components can gather provenance data during remote workflow execution.  ...  One of the main advantages of using a scientific workflow management system (SWfMS) to orchestrate data flows among scientific activities is to control and register the whole workflow execution.  ...  They are being used to orchestrate scientific data flows by controlling the whole execution of the workflow and gathering provenance data along the execution.  ... 
doi:10.1145/1646468.1646470 dblp:conf/sc/OgasawaraOCBEBCM09 fatcat:qelsrgzztbgdpagwys5jynvppu

Provenance management in Swift

Luiz M.R. Gadelha Jr., Ben Clifford, Marta Mattoso, Michael Wilde, Ian Foster
2011 Future generations computer systems  
In this article we describe these capabilities and evaluate interoperability with other systems through the use of the Open Provenance Model.  ...  The Swift parallel scripting language allows for the specification, execution and analysis of large-scale computations in parallel and distributed environments.  ...  This workflow makes extensive use of conditional and loop flow controls and database operations.  ... 
doi:10.1016/j.future.2010.05.003 fatcat:7nbxrnrbbrcnlfoc7y7b7rkwpe

CWEA: Automated log retrieval for performance analysis of service-oriented scientific workflows

Elias el Khaldi Ahanach, Zhiming Zhao
2019 Zenodo  
However, it is very challenging to analyze the workow performance, due to the diculty in gathering and analyzing perfor- mance metrics across distributed infrastructures.  ...  The prototype uses the provenance format, produced by Apache Taverna, which is a widely used workow management system in dierent elds of science.  ...  H1: Using workflow provenance to automatically collect relevant performance data, we can simplify the process of finding anomalies or bottlenecks in a workflow execution.  ... 
doi:10.5281/zenodo.3521575 fatcat:2gtpcnitvjhmrczy2w5fuugii4

Active Provenance for Data-Intensive Workflows: Engaging Users and Developers

Alessandro Spinuso, Malcolm Atkinson, Federica Magnoni
2019 2019 15th International Conference on eScience (eScience)  
We address provenance tasks such as extraction of domain metadata, injection of custom annotations, accuracy and integration of records from multiple independent workflows running in distributed contexts  ...  Provenance Types handle domain contextualisation and allow developers to model lineage patterns by re-defining API methods, composing easy-to-use extensions.  ...  Our framework extends this action with explicit provenance controls by superimposing a provenance type.  ... 
doi:10.1109/escience.2019.00077 dblp:conf/eScience/SpinusoAM19 fatcat:omeuylderba3vhqsswwahqve3q

Linking provenance with system logs: a context aware information integration and exploration framework for analyzing workflow execution

Elias el Khaldi Ahanach, Spiros Koulouzis, Zhiming Zhao
2019 Zenodo  
In this paper, we propose an architecture to automate the integration among the workflow provenance information with the performance information collected from infrastructure nodes running workflow tasks  ...  When executing scientific workflows in a distributed environment, anomalies of the workflow behavior are often caused by a mixture of different issues, e.g., careless design of the workflow logic, buggy  ...  We assume the use case in which an actor is using the GUI to gather and analyze performance data, giving workflow descriptions as input. V.  ... 
doi:10.5281/zenodo.3466765 fatcat:fp7irc3umnan3awmxigpteeiee

Editorial: Data Science Applications to Inverse and Optimization Problems in Earth Science

Olwijn Leeuwenburgh, Alexandre A. Emerick, Behnam Jafarpour, Dongxiao Zhang, Xiaodong Luo
2021 Frontiers in Applied Mathematics and Statistics  
Gao et al. present an extension of the distributed Gauss-Newton method to optimization problems that allows for large numbers of controls by use of a limited-memory BFGS scheme.  ...  workflows.  ...  In particular, they investigate if it is possible to learn a representation of the processes underlying the observed data that could be used to interpolate to times that are not directly observed.  ... 
doi:10.3389/fams.2021.756949 fatcat:2vb3a3xkxzefni5bdf5pheqqpq

LabelFlow Framework for Annotating Workflow Provenance

Pinar Alper, Khalid Belhajjame, Vasa Curcin, Carole Goble
2018 Informatics  
A stand-out feature of workflows is their ability to record provenance from executions.  ...  In this paper we investigate whether provenance can be exploited to support reporting. Specifically; we outline a case-study based on a real-world workflow and set of reporting queries.  ...  Acknowledgments: Authors would like to thank the members of the e-Science Lab at the University of Manchester for their with using Taverna workflow system's APIs.  ... 
doi:10.3390/informatics5010011 fatcat:g2wyaqo7hva55mc4qcem7lbopy

Using Context Elements and Data Provenance to Support Reuse in Scientific Software Ecosystem Platform

Lenita M. Ambrósio, José Maria N. David, Regina Braga, Fernanda Campos, Victor Ströele, Marco Antônio Araújo
2018 Proceedings of the 20th International Conference on Enterprise Information Systems  
[Conclusions] Context elements and data provenance, associated with inference mechanisms, can be used to support the reuse in scientific experimentation process.  ...  [Objectives] The goal of this paper is to present a provenance and context metadata management approach that support researchers to reuse experiments in a collaborative and distributed platform.  ...  An explicit concern with the distributed nature of the scientific experimentation process is not part of the original ProvONE ontology.  ... 
doi:10.5220/0006676302550262 dblp:conf/iceis/AmbrosioDBCSA18 fatcat:skdihljltnbg5d5mr3fulkzbze

A Distributed Algorithm for Determining the Provenance of Data

Paul T. Groth
2008 2008 IEEE Fourth International Conference on eScience  
In this paper, we describe an algorithm, D-PQuery, for determining the provenance of data from distributed sources of provenance information in a parallel fashion.  ...  As computational techniques for tracking provenance have become more widely used, applications are beginning to produce large quantities of provenance information.  ...  D-PQuery on the other hand gathers dependency data directly from the Kickstart records themselves, which may be distributed in any files accessible to the algorithm.  ... 
doi:10.1109/escience.2008.38 dblp:conf/eScience/Groth08 fatcat:co2anfe74faitiuotmlwkxtbdm

DFL designer

Jacek Sroka, Piotr Włodarczyk, Łukasz Krupa, Jan Hidders
2010 Proceedings of the 1st International Workshop on Workflow Approaches to New Data-centric Science - Wands '10  
COSW tools are used in applied sciences like bioinformatics where structured data is processed with the use of specialized services which are made available online by scientific institutions.  ...  They make such data processing experiments easier to conduct by the experimentators and easier to comprehend and repeat by the reviewers.  ...  Acknowledgments: The authors thank Lukasz Krupa and Paolo Missier for their contribution to the implementation.  ... 
doi:10.1145/1833398.1833403 fatcat:5wjwu4sln5cipdtsgamupcjmre

A Survey of Data-Intensive Scientific Workflow Management

Ji Liu, Esther Pacitti, Patrick Valduriez, Marta Mattoso
2015 Journal of Grid Computing  
A data-intensive scientific workflow is useful for modeling such process.  ...  workflows and exploit the resources distributed in different infrastructures such as grid and cloud.  ...  The provenance data is stored in a MySQL database. The Kepler Query API is used to retrieve provenance information and to display provenance graphs of workflow execution.  ... 
doi:10.1007/s10723-015-9329-8 fatcat:5urst5aphjftbli3pukmnbutri

An Identity Crisis in the Life Sciences [chapter]

Jun Zhao, Carole Goble, Robert Stevens
2006 Lecture Notes in Computer Science  
The provenance logs of workflow executions are recorded as RDF graphs. The log of one workflow run is used to trace the history of its execution process.  ...  my Grid is an e-Science project assisting life scientists to build workflows that gather data from distributed, autonomous, replicated and heterogeneous resources.  ...  The authors would like to acknowledge the other members of the my Grid team for their contributions, and in particular acknowledge Tom Oinn, Matthew Pocock, Daniele Turi and Chris Wroe.  ... 
doi:10.1007/11890850_26 fatcat:rloubmu3krfgbnpedyeufd2a7e

Sketching Distributed Data Provenance [chapter]

Tanu Malik, Ashish Gehani, Dawood Tariq, Fareed Zaffar
2013 Studies in Computational Intelligence  
The parallel operation substantially improves the performance of distributed path queries. We deployed SPADE to collect fine-grained provenance of workflows used in the NIGHTINGALE project [30] .  ...  A characteristic feature of such distributed applications is that they are often conducted in loosely controlled environments and use heterogeneous software platforms.  ...  Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.  ... 
doi:10.1007/978-3-642-29931-5_4 fatcat:gvdf4ogu4zhrphllzixqhlcpli

Representing distributed systems using the Open Provenance Model

Paul Groth, Luc Moreau
2011 Future generations computer systems  
However, to date, the ability of OPM to represent distributed systems has not been verified. In this work, we show how OPM can be used to represent a set of distributed systems' patterns.  ...  Finally, we define a contract that enables participants in a distributed system to ensure that their provenance can be integrated cohesively.  ...  Thus, provenance needs to be gathered, collated, and understood across heterogeneous systems that are physically and logically distributed.  ... 
doi:10.1016/j.future.2010.10.001 fatcat:krpp6udl3fcthd76gmm6242gum
« Previous Showing results 1 — 15 out of 6,435 results