A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Filters
Custom Execution Environments with Containers in Pegasus-enabled Scientific Workflows
[article]
2019
arXiv
pre-print
Preserved Fulltext
This paper highlights some unique challenges that may arise when using containers in distributed scientific workflows. ...
While these steps are often completely described, including the input parameters, datasets, and codes, the environment in which these steps are executed is only described at a higher level with endpoints ...
Workflows." ...
arXiv:1905.08204v1
fatcat:wz377xxoifbl5b5on3xsgu3uya
Integrating existing scientific workflow systems
2007
Proceedings of the 2nd workshop on Workflows in support of large-scale science - WORKS '07
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2010; you can also visit the original URL.
The file type is application/pdf.
Scientific workflows have become an important tool used by scientists to conduct large-scale analysis in distributed environments. ...
In this paper we examine the possibility of integrating two well-known workflow systems Kepler and Pegasus and examine the opportunities and challenges presented by such an integration. ...
Catalog Integration As we have shown in Figure 4b , we have integrated the Pegasus Transformation Catalog with the Kepler environment in a way which allows the user to customize the Pegasus actor. ...
doi:10.1145/1273360.1273365
dblp:conf/hpdc/MandalDMSV07
fatcat:zqpuaz46mraj5ofavx5cgjdjoy
Enabling large-scale scientific workflows on petascale resources using MPI master/worker
2012
Proceedings of the 1st Conference of the Extreme Science and Engineering Discovery Environment on Bridging from the eXtreme to the campus and beyond - XSEDE '12
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2012; you can also visit the original URL.
The file type is application/pdf.
We describe how the partitioning and job management has been implemented in the Pegasus Workflow Management System. ...
In this paper we describe a new approach to executing large, fine-grained workflows on distributed petascale systems. ...
The work described in this paper used the Extreme Science and Engineering Discovery Environment (XSEDE), which is supported by National Science Foundation grant number OCI-1053575. ...
doi:10.1145/2335755.2335846
fatcat:tl3il2esvncnvlf7w2epbrok54
Pegasus, a workflow management system for science automation
2015
Future generations computer systems
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2018; you can also visit the original URL.
The file type is application/pdf.
This paper provides an integrated view of the Pegasus system, showing its capabilities that have been developed over time in response to application needs and to the evolution of the scientific computing ...
The paper describes how Pegasus achieves reliable, scalable workflow execution across a wide variety of computing infrastructures. ...
Pegasus has been in development since 2001 and has benefited greatly from the expertise and efforts of people who worked on it over the years. ...
doi:10.1016/j.future.2014.10.008
fatcat:u5lbouuekvduhfdvdi7phfyn7i
Experiences using cloud computing for a scientific workflow application
2011
Proceedings of the 2nd international workshop on Scientific cloud computing - ScienceCloud '11
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2016; you can also visit the original URL.
The file type is application/pdf.
in a cloud environment. ...
Clouds are rapidly becoming an important platform for scientific applications. In this paper we describe our experiences running a scientific workflow application in the cloud. ...
This enables workflows to be automatically adapted to run across different environments. ...
doi:10.1145/1996109.1996114
fatcat:xaldzcbzrjdvzob4qjglgufkvq
Bringing Scientific Workflow to the Masses via Pegasus and HUBzero
2013
International Workshop on Science Gateways
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
With the press of a button, these tools can launch Pegasus workflows on national computing infrastructures and bring results back for plotting and visualization. ...
The Pegasus Workflow Management System can manage workflows comprised of millions of tasks, all the while recording data about the execution and intermediate results so that the provenance of the final ...
Customized Workflows in Pegasus More experienced users can learn how to create their own customized Pegasus workflows. ...
dblp:conf/iwsg/McLennanCMDRVKS13
fatcat:dpkhztkt7fh3vndvv6ubxclqny
Executing cyclic scientific workflows in the cloud
2021
Journal of Cloud Computing: Advances, Systems and Applications
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
AbstractWe present an algorithm and a software architecture for a cloud-based system that executes cyclic scientific workflows whose structure may change during run time. ...
The evaluation demonstrates that our algorithm is able to execute dynamically changing workflows with cycles and that design and maintenance of complex workflows is easier than with existing solutions. ...
Pegasus and Argo require custom actions, sub-workflows/templates, or counter variables to simulate this behaviour. ...
doi:10.1186/s13677-021-00229-7
fatcat:ogzdrrdidza6zols2yb5sg3fsq
Evaluating I/O aware network management for scientific workflows on networked clouds
2013
Proceedings of the Third International Workshop on Network-Aware Data Management - NDM '13
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2018; you can also visit the original URL.
The file type is application/pdf.
The application was deployed on a virtualized HTCondor environment provisioned dynamically from the ExoGENI networked cloud testbed, and managed by the Pegasus workflow manager. ...
This paper presents a performance evaluation of scientific workflows on networked cloud systems with particular emphasis on evaluating the effect of provisioned network bandwidth on application I/O performance ...
The recent advances in enabling on-demand network circuits, coupled with programmable edge technologies create an unprecedented opportunity to enable complex data-intensive scientific applications to run ...
doi:10.1145/2534695.2534698
dblp:conf/sc/MandalRBXCRD13
fatcat:b4egdprs6nb7xme4codm2k4qou
Pegasus: A Framework for Mapping Complex Scientific Workflows onto Distributed Systems
2005
Scientific Programming
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2007; you can also visit the original URL.
The file type is application/pdf.
Pegasus enables users to represent the workflows at an abstract level without needing to worry about the particulars of the target execution systems. ...
This paper describes the Pegasus framework that can be used to map complex scientific workflows onto distributed resources. ...
Acknowledgments Pegasus is supported by NSF under grants ITR-0086044 (GriPhyN), ITR AST0122449 (NVO) and EAR-0122464 (SCEC/ITR). ...
doi:10.1155/2005/128026
fatcat:pld7bbr5fjgtxpmlewtgtzd4km
Enhanced Resource Provisioning Strategies for Scientific Workflows in Cloud Environment: A Survey
2018
International Journal of Computer Applications
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
The on-demand provisioning and resource availability in cloud computing make it ideal for executing scientific workflow applications. To ensure better performance, there is a need for auto-scaling. ...
The problem of assigning resources to tasks and orchestrating their execution to preserve the dependencies of workflows is NP-complete. [2] Hence, no optimal solution can be found in polynomial time. ...
The IWD is a population based algorithm proposed by Shah-Hosseini in 2007 for solving combinatorial problems. The IWD algorithm mimics the dynamics of river system and the ...
doi:10.5120/ijca2018917097
fatcat:sbdiqpmbxvd6njvdg6xwyeu7iu
Simplifying construction of complex workflows for non-expert users of the Southern California Earthquake Center Community Modeling Environment
2005
SIGMOD record
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL.
The file type is application/pdf.
However, non-expert users are better served with simple interfaces that abstract away system complexities and still enable them to construct and execute complex workflows. ...
Workflow systems often present the user with rich interfaces that express all the capabilities and complexities of the application programs and the computing environments that they support. ...
Environment (SCEC/CME): An Information Infrastructure for System-Level Earthquake Research). ...
doi:10.1145/1084805.1084811
fatcat:o33zte5zarecxlmve3rjrisq2e
From Data to Knowledge to Discoveries: Artificial Intelligence and Scientific Workflows
2009
Scientific Programming
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2008; you can also visit the original URL.
The file type is application/pdf.
A key emerging concept is scientific workflows, which provide a declarative representation of complex scientific applications that can be automatically managed and executed in distributed shared resources ...
Scientific computing has entered a new era of scale and sharing with the arrival of cyberinfrastructure facilities for computational experimentation. ...
The author would like to thank Ewa Deelman and Carole Goble for very useful discussions on scientific workflows.
Bibliography ...
doi:10.1155/2009/167604
fatcat:ydfv3sqmkjdwxj4dxmimkdid2e
Comparing FutureGrid, Amazon EC2, and Open Science Grid for Scientific Workflows
2013
Computing in science & engineering (Print)
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2018; you can also visit the original URL.
The file type is application/pdf.
In this paper we describe our experiences modeling the periodogram application as a scientific workflow using Pegasus, and deploying it on the FutureGrid scientific cloud testbed, the Amazon EC2 commercial ...
As a result, workflows enable scientists to easily define multistage computational and data processing pipelines that can be executed in parallel on distributed resources, which automates complex analyses ...
This research will aid in the development of capabilities for science applications in the cloud that cannot be provided by traditional scientific computing platforms such as grids. ...
doi:10.1109/mcse.2013.44
fatcat:loobib3hfbh63l3xuevxdpvyue
Cloud infrastructure provenance collection and management to reproduce scientific workflows execution
2018
Future generations computer systems
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Note that this fulltext copy is not of the "primary" version of this work. The version it corresponds to is:
2018 pre-print arXiv:1803.06867v1 fatcat:uu4phq2j2bc2plznfwrvif2m7i
Therefore, a mechanism is required that enables capturing of infrastructure information along with the provenance of workflows executing on the Cloud to facilitate the re-creation of execution environment ...
The emergence of Cloud computing provides a new computing paradigm for scientific workflow execution. ...
Acknowledgements The authors would like to acknowledge the support of the European Union in funding this work via the neuGRID4You (N4U) project (grant agreement n. 283562, 2011-2015). ...
doi:10.1016/j.future.2017.07.015
fatcat:mmzzhovmqbhhtovcvq2ztp2c6i
Parameterized specification, configuration and execution of data-intensive scientific workflows
2010
Cluster Computing
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2012; you can also visit the original URL.
The file type is application/pdf.
Data analysis processes in scientific applications can be expressed as coarse-grain workflows of complex data processing operations with data flow dependencies between them. ...
are known to affect their execution times. ...
Acknowledgements This research was supported in part by the National Science Foundation under Grants #CNS-0403342, #CNS-0426241, #CSR-0509517, #CSR-0615412, #ANI-0330612, #CCF-0342615, #CNS-0203846, #ACI-b0130437 ...
doi:10.1007/s10586-010-0133-8
pmid:22623878
pmcid:PMC3356923
fatcat:otnq42fe2nf7rge64b4j3cvaw4
« Previous
Showing results 1 — 15 out of 543 results