Filters








6,847 Hits in 3.5 sec

Integrating Domain-Data Steering with Code-Profiling Tools to Debug Data-Intensive Workflows

Vítor Silva Sousa, Leonardo Neves, Renan Souza, Alvaro L. G. A. Coutinho, Daniel de Oliveira, Marta Mattoso
2016 International Conference for High Performance Computing, Networking, Storage and Analysis  
In this paper, we propose to couple a workflow monitoring data approach to parallel code-profiling tools for workflow executions.  ...  Several parallel code-profiling tools already support performance analysis, such as Tuning and Analysis Utilities (TAU) or provide fine-grained performance statistics such as the System Activity Report  ...  In data-intensive workflows this lack becomes really an issue.  ... 
dblp:conf/sc/SousaNSC0M16 fatcat:yfzyji7rpbdqpnve4ywfydl4vi

Enhancing Petrographic Analysis Through Data Fusion

Nicholas Vito, Connor Burt, Eric Goergen
2017 Microscopy and Microanalysis  
Despite these limitations, the wealth of data extracted from BSE images provides the contextual backdrop for the workflow provided in this presentation.  ...  These characteristics have large impacts on accurately assessing porosity and permeability profiles for upscaling into reservoir models.  ...  Despite these limitations, the wealth of data extracted from BSE images provides the contextual backdrop for the workflow provided in this presentation.  ... 
doi:10.1017/s1431927617011485 fatcat:tlvsvu6kmjedhfgnwkriopu6ty

Profiling Resource Utilization of Bioinformatics Workflows [article]

Huazeng Deng, Ling-Hong Hung, Raymond Schooley, David Perez, Niharika Arumilli, Ka Yee Yeung, Wes Lloyd
2020 arXiv   pre-print
The Container Profiler can produce utilization snapshots at multiple time points, allowing for continuous monitoring of the resources consumed by a container workflow.  ...  We examined the collected profile metrics and confirmed that they were consistent with the expected CPU, disk, network resource utilization patterns for the different stages of the workflow.  ...  We acknowledge support from the AWS Cloud Credits for Research (awarded to LHH, WL, and KYY).  ... 
arXiv:2005.11491v1 fatcat:2d2hxcnvmbdbde4jpt34ctn3jy

Serverless is More: From PaaS to Present Cloud Computing

Erwin van Eyk, Lucian Toader, Sacheendra Talluri, Laurens Versluis, Alexandru Uta, Alexandru Iosup
2018 IEEE Internet Computing  
Data-intensive applications Fine-grained data-centric programming models.  ...  We envision for the future the design and implementation of fine-grained, data-centric, serverless programming models.  ... 
doi:10.1109/mic.2018.053681358 fatcat:niayqqghojcedlr6yppfacqftm

Community Resources for Enabling Research in Distributed Scientific Workflows

Rafael Ferreira da Silva, Weiwei Chen, Gideon Juve, Karan Vahi, Ewa Deelman
2014 2014 IEEE 10th International Conference on e-Science  
All of the tools and data are freely available online for the community at http://www.workflowarchive.org. These data have already been leveraged for a number of studies.  ...  In this paper we describe a collection of tools and data that have enabled research in new techniques, algorithms, and systems for scientific workflows.  ...  Energy-Efficiency: We have used WorkflowSim to simulate a controlled distributed environment for profiling and analyzing energy-efficiency in data-intensive scientific workflows [25] .  ... 
doi:10.1109/escience.2014.44 dblp:conf/eScience/SilvaCJVD14 fatcat:bnmiqfstnrbpzavs52aklwj5ju

Characterizing and profiling scientific workflows

Gideon Juve, Ann Chervenak, Ewa Deelman, Shishir Bharathi, Gaurang Mehta, Karan Vahi
2013 Future generations computer systems  
The characterization is based on novel workflow profiling tools that provide detailed information about the various computational tasks that are present in the workflow.  ...  The study also uncovered inefficiency in a workflow component implementation, where the component was re-reading the same data multiple times.  ...  Kickstart does not, however, collect fine-grained profiling data. One system that does collect fine-grained profiles is the ParaTrac system [60] .  ... 
doi:10.1016/j.future.2012.08.015 fatcat:ljsx7tntkjgg5ewbfkofh2szlq

SysOptic: A Fine-Grained Monitoring System for Virtual Machines Based on PMU

Pin Liu, Renyu Yang, Jie Sun, Xudong Liu
2019 2019 IEEE International Conference on Service-Oriented System Engineering (SOSE)  
Performance Monitoring Unit on CPU (PMU) can obtain fine-grained monitoring data by adopting interrupt sampling method based on hardware events.  ...  In this paper, we present a fine-grained monitoring system SysOptic based on PMU virtualization.  ...  Numerous performance analysis and profiler tools can analyze fine-grained monitoring data through PMU hardware at process level or function level.  ... 
doi:10.1109/sose.2019.00042 dblp:conf/sose/LiuYSL19 fatcat:5xhm3465zjhalfychl7tb5qowq

Big Data Provenance: Challenges and Implications for Benchmarking [chapter]

Boris Glavic
2014 Lecture Notes in Computer Science  
Provenance has been studied by the database, workflow, and distributed systems communities, but provenance for Big Data -let us call it Big Provenance -is a largely unexplored field.  ...  Such information is useful for debugging data and transformations, auditing, evaluating the quality of and trust in data, modelling authenticity, and implementing access control for derived data.  ...  Coarse-grained provenance has been studied intensively by the workflow community.  ... 
doi:10.1007/978-3-642-53974-9_7 fatcat:zjoio7iemre57ktgvzzssqs7lm

Workflow Performance Profiles: Development and Analysis [chapter]

Dariusz Król, Rafael Ferreira da Silva, Ewa Deelman, Vickie E. Lynch
2017 Lecture Notes in Computer Science  
This paper presents a method for performance profiles development of scientific workflow.  ...  The workflow executes an ensemble of molecular dynamics and neutron scattering intensity calculations to optimize a model parameter value.  ...  Therefore, we see the need for fine-grained monitoring tools to automatically collect such information, and to build workflow profiles.  ... 
doi:10.1007/978-3-319-58943-5_9 fatcat:xity6p7b35agbg53ifqqombggy

Task Runtime Prediction in Scientific Workflows Using an Online Incremental Learning Approach

Muhammad Hafizhuddin Hilman, Maria Alejandra Rodriguez, Rajkumar Buyya
2018 2018 IEEE/ACM 11th International Conference on Utility and Cloud Computing (UCC)  
To improve the performance of the predictions, we harness fine-grained resources monitoring data in the form of time-series records of CPU utilization, memory usage, and I/O activities that are reflecting  ...  With the emergence of multi-tenant Workflow as a Service (WaaS) platforms that use clouds for deploying scientific workflows, task runtime prediction becomes more challenging because it requires the processing  ...  ACKNOWLEDGMENTS This research is partially supported by LPDP (Indonesia Endowment Fund for Education) and ARC (Australia Research Council) research grant.  ... 
doi:10.1109/ucc.2018.00018 dblp:conf/ucc/HilmanRB18 fatcat:swkvuqb7cbddhiyl4hmtvb46pe

Domain-specific summarization of Life-Science e-experiments from provenance traces

Alban Gaignard, Johan Montagnat, Bernard Gibaud, Germain Forestier, Tristan Glatard
2014 Journal of Web Semantics  
With the unprecedented growth of Life-Science data repositories, identifying relevant data for analysis becomes increasingly difficult.  ...  opportunities in line with Linked Data initiatives.  ...  We would also like to thank Olivier Corby and Catherine Faron Zucker for their support and advises regarding the Corese/KGRAM Semantic Web engine.  ... 
doi:10.1016/j.websem.2014.07.001 fatcat:imva7ugy7zdo7c6rjnpbzwpyri

Run-time Parameter Sensitivity Analysis Optimizations [article]

Eduardo Scartezini, Willian Barreiros Jr., Tahsin Kurc, Jun Kong, Alba C. M. A. Melo, Joel Saltz, George Teodoro
2019 arXiv   pre-print
The execution of the application is rather compute intensive, and a SA requires it to process the input data multiple times as parameter values are systematically varied.  ...  Efficient execution of parameter sensitivity analysis (SA) is critical to allow for its routinely use.  ...  This raises possibilities for reuse of stages (coarse-grain) and tasks (fine-grain).  ... 
arXiv:1910.14548v1 fatcat:bxygwno5a5fxtlwsxvg243hohe

CAST

Yue Cheng, M. Safdar Iqbal, Aayush Gupta, Ali R. Butt
2015 Proceedings of the 24th International Symposium on High-Performance Parallel and Distributed Computing - HPDC '15  
The approach takes the first step towards providing storage tiering support for data analytics in the cloud.  ...  Cast performs offline workload profiling to construct job performance prediction models on different cloud storage services, and combines these models with workload specifications and high-level tenant  ...  We also thank Taha Hasan for helpful discussions on the idea in its early stages, and Krishnaraj K. Ravindranathan for help in improving the presentation.  ... 
doi:10.1145/2749246.2749252 dblp:conf/hpdc/ChengIGB15 fatcat:zzwsi524tnbehkonden4hbs3vy

How Much Energy Can Green HPC Cloud Users Save?

David Guyon, Anne-Cecile Orgerie, Chrtistine Morin, Deb Agarwal
2017 2017 25th Euromicro International Conference on Parallel, Distributed and Network-based Processing (PDP)  
However, data centers hosting cloud systems consume enormous amounts of energy. Reducing this consumption becomes an urgent challenge with the rapid growth of cloud utilization.  ...  In this paper, we explore a way for energy-aware HPC cloud users to reduce their footprint on cloud infrastructures by reducing the size of the virtual resources they are asking for.  ...  The electrical consumption of the servers has been measured thanks to the fine-grained wattmeters available on them [11] .  ... 
doi:10.1109/pdp.2017.62 dblp:conf/pdp/GuyonOMA17 fatcat:icvwduivnfbwjbk5agh6lhrwmu

Software defined architectures for data analytics

Vito Giovanni Castellana, Marco Minutoli, Antonino Tumeo, Marco Lattuada, Pietro Fezzardi, Fabrizio Ferrandi
2019 Proceedings of the 24th Asia and South Pacific Design Automation Conference on - ASPDAC '19  
However, their fine-grained nature still leads to issues for the design software and still makes dynamic reconfiguration impractical.  ...  In this position paper, we describe a possible toolchain for reconfigurable architectures targeted at data analytics.  ...  Current fine-grained FP-GAs have already demonstrated success in accelerating machine learning, and memory-intensive workloads, including graph algorithms.  ... 
doi:10.1145/3287624.3288754 dblp:conf/aspdac/CastellanaMTLFF19 fatcat:ip4n6z5ghzdubmzs7g6vsq3jmu
« Previous Showing results 1 — 15 out of 6,847 results