383,703 Hits in 6.4 sec

Integrated Data and Task Management for Scientific Applications [chapter]

Jarek Nieplocha, Sriram Krishamoorthy, Marat Valiev, Manoj Krishnan, Bruce Palmer, P. Sadayappan
2008 Lecture Notes in Computer Science  
Several emerging application areas require intelligent management of distributed data and tasks that encapsulate execution units for collection of processors or processor groups.  ...  This paper describes an integration of data and task parallelism to address the needs of such applications in context of the Global Array (GA) programming model.  ...  Our task management system can handle variable sized processor groups. This capability was used for a quantum mechanical application with complex data and task management requirements.  ... 
doi:10.1007/978-3-540-69384-0_6 fatcat:vrqom67e2ncuzbr27vgrmkyug4

Opportunities and Challenges in Running Scientific Workflows on the Cloud

Yong Zhao, Xubo Fei, Ioan Raicu, Shiyong Lu
2011 2011 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery  
We coin the term "Cloud Workflow", to refer to the specification, execution, provenance tracking of large-scale scientific workflows, as well as the management of data and computing resources to enable  ...  Cloud computing; Scientific Workflow; Cloud workflow; Data Intensive Computing I.  ...  Data management challenge As scientific applications become more data intensive, the management of data resources and dataflow between the storage and compute resources is becoming the main bottleneck.  ... 
doi:10.1109/cyberc.2011.80 dblp:conf/cyberc/ZhaoFRL11 fatcat:zlpe7p73lnh2jksgtnstvkafn4

Enabling scalable scientific workflow management in the Cloud

Yong Zhao, Youfu Li, Ioan Raicu, Shiyong Lu, Wenhong Tian, Heng Liu
2015 Future generations computer systems  
Scientific workflow management plays a very important role for scientific computing and application coordination, while Cloud computing offers scalability and resource on-demand.  ...  We devise autonomous methods to integrate scientific workflow management systems with Cloud platforms and also provision resources for large scale workflows, which can facilitate scientists to easily manage  ...  Acknowledgments This work was supported by National Natural Science Foundation of China No. 61272528, No. 61034005, and the Central University Fund (ID-ZYGX2013J073).  ... 
doi:10.1016/j.future.2014.10.023 fatcat:myqbnunkk5dmdml4stdgvedk2y

Scientific Workflow Composition in Heterogeneous Environments

Jamal Alhiyafi, Shiyong Lu, Jeffrey L. Ram
2012 International Journal of Intelligent Information Processing  
technique for parallel executing of heterogeneous scientific workflows; and 3) to present a GENOEMFLOW based scientific workflow application to showcase the application of our system.  ...  The advantages of high-performance grid-based computing for scientific workflows, and the problems noted with existing scientific workflow management software, provide the rationale for the development  ...  Also, VIEW is the first system that supports dataflow-based MapReduce-style scientific workflows for data-intensive scientific applications [16] , and it supports an innovative task abstraction and mapping  ... 
doi:10.4156/ijiip.vol3.issue1.3 fatcat:qikhy7bk3nhafobwbxph2m3fmi

Scalable Scientific Workflows Management System SWFMS

M. Abdul
2016 International Journal of Advanced Computer Science and Applications  
Furthermore, the involvement of increasingly large number of scientific instruments such as sensors and machines makes the scientific data management even more challenging since the data generated from  ...  data products processed by these applications.  ...  Application Management and Data Management.  ... 
doi:10.14569/ijacsa.2016.071137 fatcat:eioqjgh4yndt5inlv2ajki2p3y

Scientific workflows management and scheduling in cloud computing: Taxonomy, prospects, and challenges

Zulfiqar Ahmad, Ali Imran Jehangiri, Mohammed Alaa Ala'anzy, Mohamed Othman, Rohaya Latip, Sardar Khaliq uz Zaman, Arif Iqbal Umar
2021 IEEE Access  
The services provided by cloud computing are broadly used for business and scientific applications. Business applications are task oriented applications and structured into business workflows.  ...  Whereas, scientific applications are data oriented and compute intensive applications and structured into scientific workflows.  ...  ACKNOWLEDGMENT The authors would like to thank the financial support and facilities provided by Universiti Putra Malaysia and the Ministry of Education Malaysia for the execution, completion and publication  ... 
doi:10.1109/access.2021.3070785 fatcat:nbrkndnhzra57lpbhzpyeht3wq

Architecting Cloud Workflow: Theory and Practice

Yong Zhao, Youfu Li, Ioan Raicu, Shiyong Lu, Xuan Zhang
2014 2014 IEEE International Conference on Computer and Information Technology  
The data scale, science analysis and processing complexity in scientific community are growing exponentially in the "big data" era.  ...  Cloud computing paradigm has been widely adopted to provide unprecedented scalability and resources on demand, while scientific workflow management systems (SWFMSs) have been proven essential to scientific  ...  ACKNOWLEGMENT This paper is supported by the National Science Foundation of China No. 61034005 and No. 61272528.  ... 
doi:10.1109/cit.2014.81 dblp:conf/IEEEcit/ZhaoLRLZ14 fatcat:4idyjk4rwfc7rbn2ynmv2avfiu

Migrating Scientific Workflow Management Systems from the Grid to the Cloud [chapter]

Yong Zhao, Youfu Li, Ioan Raicu, Cui Lin, Wenhong Tian, Ruini Xue
2014 Cloud Computing for Data-Intensive Applications  
At the same time, scientific workflow management systems provide essential support and functionality to scientific computing, such as management of data and task dependencies, job scheduling and execution  ...  cluster provisioning, high-throughput task scheduling and execution, and efficient and scalable resource management in the Cloud.  ...  Acknowledgments This paper is supported by the key project of National Science Foundation of China No. 61034005 and No. 61272528.  ... 
doi:10.1007/978-1-4939-1905-5_10 fatcat:2hx7wzsucvdehdmr4fccd3yd2y

Devising a Cloud Scientific Workflow Platform for Big Data

Yong Zhao, Youfu Li, Shiyong Lu, Ioan Raicu, Cui Lin
2014 2014 IEEE World Congress on Services  
In this paper, we first discuss the challenges for scientific workflow applications and the available solutions in details, and analyze the essential requirements for a scientific computing Cloud platform  ...  Scientific workflow management systems (SWFMSs) are facing unprecedented challenges from big data deluge.  ...  to scientific computing, including management of data and task dependencies, job scheduling and execution, provenance tracking, etc.  ... 
doi:10.1109/services.2014.75 dblp:conf/services/ZhaoLLRL14 fatcat:q3ksjwm6dzd3zbmluwbqy6bun4

Reducing Complexity in Management of eScience Computations

Per-Olov Östberg, Andreas Hellander, Brian Drawert, Erik Elmroth, Sverker Holmgren, Linda Petzold
2012 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (ccgrid 2012)  
for computational science -software systems that abstract and virtualize the processes of managing scientific computations on heterogeneous distributed resource systems.  ...  We explore an approach based on separation of computation design (application development) and distributed execution of computations, and investigate best practices for construction of virtual infrastructures  ...  The authors acknowledge MikaelÖhman, Sebastian Gröhn, and Anders Häggström for work related to the project, and the anonymous reviewers for valuable feedback and interesting discussions.  ... 
doi:10.1109/ccgrid.2012.72 dblp:conf/ccgrid/OstbergHDEHP12 fatcat:ofeeyjizkzhevfodvx4ofx6z7i

Service-Oriented Architecture for VIEW: A Visual Scientific Workflow Management System

Cui Lin, Shiyong Lu, Zhaoqiang Lai, Artem Chebotko, Xubo Fei, Jing Hua, Farshad Fotouhi
2008 2008 IEEE International Conference on Services Computing  
These requirements demand a new architectural design for scientific workflow management systems (SWFMSs).  ...  Scientific workflows have recently emerged as a new paradigm for scientists to formalize and structure complex and distributed scientific processes to enable and accelerate many scientific discoveries.  ...  data product, service, and application management.  ... 
doi:10.1109/scc.2008.118 dblp:conf/IEEEscc/LinLLCFHF08 fatcat:drwsybovbrgqfczphkwj7latem

A characterization of workflow management systems for extreme-scale applications

Rafael Ferreira da Silva, Rosa Filgueira, Ilia Pietri, Ming Jiang, Rizos Sakellariou, Ewa Deelman
2017 Future generations computer systems  
Over the last years, scientific workflows have been established as an important abstraction that captures data processing and computation of large and complex scientific applications.  ...  The paper also surveys workflow applications and identifies gaps for future research on the road to extreme-scale workflows and management systems.  ...  In addition, as the scientific community prepares for extreme-scale computing, big data analytics is becoming an essential part of the scientific process for insights and discoveries [25] .  ... 
doi:10.1016/j.future.2017.02.026 fatcat:3ri627jyy5a47hgh5r343f5ife

Collaborative Scientific Workflows

Shiyong Lu, Jia Zhang
2009 2009 IEEE International Conference on Web Services  
In recent years, a number of scientific workflow management systems (SWFMSs) have been developed to help domain scientists synergistically integrate distributed computations, datasets, and analysis tools  ...  to enable and accelerate scientific discoveries.  ...  Second, provenance data management for the reproducibility of scientific results is essential for scientific workflows but not for business workflows.  ... 
doi:10.1109/icws.2009.150 dblp:conf/icws/LuZ09 fatcat:uixs7xfwond7ncnairul5yheje

Provenance Query Patterns for Many-Task Scientific Computing

Luiz M. R. Gadelha Jr., Marta Mattoso, Michael Wilde, Ian T. Foster
2011 Workshop on the Theory and Practice of Provenance  
We present a set of patterns that can be identified in these queries, which is being used as a basis for the design and implementation of a provenance management system for many-task scientific computations  ...  They allow for one to determine how each resulting data set was derived from other data sets and applications.  ...  Acknowledgments This work was supported in part by CAPES, CNPq, and the Office of Advanced Scientific Computing Research, Office of Science, U.S.  ... 
dblp:conf/tapp/GadelhaMWF11 fatcat:q42ogoviibbm3dvpwb2hmghkmy

Middleware Building Blocks for Workflow Systems [article]

Matteo Turilli, Vivek Balasubramanian, Andre Merzky, Ioannis Paraskevakos, Shantenu Jha
2019 arXiv   pre-print
This paper offers three main contributions: (i) showing the relevance of the design principles of self-sufficiency, interoperability, composability and extensibility for middleware to support scientific  ...  Four case studies are presented, discussing how RADICAL-Cybertools are integrated with existing workflow, workload, and general purpose computing systems to support the execution of scientific workflows  ...  We thank Daniel Smith, Levi Naden and Sam Ellis (MolSSI) for useful discussions and insight. This was was supported primarily by NSF 1440677 and DOE ASCR de-sc0016280.  ... 
arXiv:1903.10057v1 fatcat:vvlkcxkuebhdxcp5bcqrcwu4tu
« Previous Showing results 1 — 15 out of 383,703 results