14,853 Hits in 9.0 sec

Improving the Scalability of Parallel Jobs by adding Parallel Awareness to the Operating System

Terry Jones, Paul Tomlinson, Mark Roberts, Shawn Dawson, Rob Neely, William Tuel, Larry Brenner, Jeffrey Fier, Robert Blackmore, Patrick Caffrey, Brian Maskell
2003 Proceedings of the 2003 ACM/IEEE conference on Supercomputing - SC '03  
and run-time system, and present a set of empirical results comparing the technique with traditional operating system scheduling.  ...  We present a novel co-scheduling scheme for improving performance of fine-grain collective activities such as barriers and reductions, describe an implementation consisting of operating system kernel modifications  ...  Adding Parallel Awareness to Intra-Node Scheduling While the AIX operating system is able to run work simultaneously on multiple processors, it is not designed to start work simultaneously on multiple  ... 
doi:10.1145/1048935.1050161 dblp:conf/sc/JonesDNTBFBCMTR03 fatcat:utaonbdorveyrjgz4h2rvitcd4

Morsel-driven parallelism

Viktor Leis, Peter Boncz, Alfons Kemper, Thomas Neumann
2014 Proceedings of the 2014 ACM SIGMOD international conference on Management of data - SIGMOD '14  
Further, the dispatcher is aware of data locality of the NUMA-local morsels and operator state, such that the great majority of executions takes place on NUMA-local memory.  ...  A third problem faced by many-core architectures is the decentralization of memory controllers, which leads to Non-Uniform Memory Access (NUMA).  ...  Abundant research in the 1990s into parallel processing led the majority of database systems to adopt a form of parallelism inspired by the Volcano [12] model, where operators are kept largely unaware  ... 
doi:10.1145/2588555.2610507 dblp:conf/sigmod/LeisBK014 fatcat:l2s367jiwfggnleamjxic2gcgq

Multilevel Threshold Secret Sharing Scheme to Secure MapReduce Computations in Cloud Computing Environment

M. Eyalarasi, P. Shanthi, A. Umamakeswari
2016 Indian Journal of Science and Technology  
A common problem is whether MapReduce could be customized to get a scalable system, when the jobs are split and reduced.  ...  Also most of the systems do not consider the issue of security in MapReduce phases.  ...  Their work improved the framework by adding a merge phase. This method shows relational algebra operations and also constructs various join algorithms.  ... 
doi:10.17485/ijst/2016/v9i48/107968 fatcat:3brujd5ulrb77oelckcvgyzwra


Christian Engelmann, P. Sadayappan, Stephen L. Scott, David E. Bernholdt, Narasimha R. Gottumukkala, Chokchai Leangsuksun, Jyothish Varma, Chao Wang, Frank Mueller, Aniruddha G. Shet
2006 ACM SIGOPS Operating Systems Review  
by exploring the use of advanced monitoring and adaptation to improve application performance and predictability of system interruptions, and by advancing computer reliability, availability and serviceability  ...  Conceptually, the MOLAR [15] research has the following goals to address these issues. • Explore the use of advanced monitoring and adaptation to improve application performance and predictability of system  ...  ACKNOWLEDGEMENTS The authors would like to thank Professor D.K.  ... 
doi:10.1145/1131322.1131337 fatcat:ihpzd2m7vffbdeznsh6upvvoey

A Survey On Data-Centric And Data-Aware Techniques For Large Scale Infrastructures

Silvina Caíno-Lores, Jesús Carretero
2016 Zenodo  
Currently, large scale data-intensive applications are hindered by the high latencies that result from the access to vastly distributed data.  ...  Recent works have suggested that improving data locality is key to move towards exascale infrastructures efficiently, as solutions to this problem aim to reduce the bandwidth consumed in data transfers  ...  As parallel file systems can significantly improve locality if designed with such purpose, there are examples of no-SQL database systems that aim to improve the overall efficiency of the data analysis  ... 
doi:10.5281/zenodo.1112257 fatcat:4l7qjgwdcrffddnwgb4dd3miuu

Measuring Software Systems Scalability for Proactive Data Center Management [chapter]

Nuno A. Carvalho, José Pereira
2010 Lecture Notes in Computer Science  
We validate the approach first using simulation and then in an actual system. The resulting better prediction of scalability properties should allow improved (self-)management practices.  ...  In this paper we address this challenge with a tool to measure the software scalability regarding CPU availability, to predict system behavior in face of varying resources and an increasing workload.  ...  time of jobs in the system is given by Eq. 17.  ... 
doi:10.1007/978-3-642-16949-6_11 fatcat:u3cekxzgvraojin32bcdnck2ye

Workflow optimization of performance and quality of service for bioinformatics application in high performance computing

Rashid Al-Ali, Nagarajan Kathiresan, Mohammed El Anbari, Eric R. Schendel, Tariq Abu Zaid
2016 Journal of Computational Science  
Additionally, most of the HPC systems are operated in a multi-user (or multi-job) environment; thus, Quality of Service (QoS) methods are essential for balancing between application performance, scalability  ...  up to 200%, parallel efficiency improved up to 39% and overall system utilization increased up to 38%.  ...  Acknowledgements The authors would like to thank Ramzi Temanni, Hakeem Almabrazi, Najeeb Syed and other Sidra Bioinformatics researchers for providing helpful comments and suggestion for running the BWA-MEM  ... 
doi:10.1016/j.jocs.2016.03.005 fatcat:5ujoe3km3fenplo3gzqirqyggu

Design of a scalable InfiniBand topology service to enable network-topology-aware placement of processes

H. Subramoni, S. Potluri, K. Kandalla, B. Barth, J. Vienne, J. Keasler, K. Tomko, K. Schulz, A. Moody, D. K. Panda
2012 2012 International Conference for High Performance Computing, Networking, Storage and Analysis  
Micro-benchmark level evaluations show that the proposed network-topology-aware MPI library can improve the latency for all message sizes by up to 40%.  ...  across the InfiniBand fabric. topology-aware MPI library on the performance of parallel scientific applications?  ...  We have demonstrated the applicability and scalability of our service by using it to design a network-topology-aware MPI library, and we have shown that its use can improve performance of real-world applications  ... 
doi:10.1109/sc.2012.47 dblp:conf/sc/SubramoniPKBVKTSMP12 fatcat:dd5mkhn4drdyrjsht7ti4ka7ne

Author Index

2008 2008 IEEE International Symposium on Parallel and Distributed Processing  
of a National Health Information System on Childhood Cancer Hirano, Akio Saving 200kW and 200K/year by Power-aware Job/Machine Scheduling Ho, Justin C.  ...  Wireless Mesh Networks Hikita, Junichi Saving 200kW and 200K/year by Power-aware Job/Machine Scheduling Hill, James H.  ... 
doi:10.1109/ipdps.2008.4536576 fatcat:7unikf5ywjhjtdd6xtrmcom3gq


Rong Chen, Haibo Chen, Binyu Zang
2010 Proceedings of the 19th international conference on Parallel architectures and compilation techniques - PACT '10  
The prevalence of chip multiprocessor opens opportunities of running data-parallel applications originally in clusters on a single machine with many cores.  ...  Based on Tiled-MapReduce, we design and implement several optimizing techniques targeting multicore, including the reuse of input and intermediate data structure among sub-jobs, a NUCA/NUMA-aware scheduler  ...  Based on the observation, this paper proposed Tiled-MapReduce, that uses the "tiling strategy" to partition a large MapReduce job into a number of small sub-jobs and handles the sub-jobs iteratively.  ... 
doi:10.1145/1854273.1854337 dblp:conf/IEEEpact/ChenCZ10 fatcat:jnbz757shbfrxjjzwtouci6key

SCALER: Scalable parallel file write in HDFS

Xi Yang, Yanlong Yin, Hui Jin, Xian-He Sun
2014 2014 IEEE International Conference on Cluster Computing (CLUSTER)  
Two camps of file systems exist: parallel file systems designed for conventional high performance computing (HPC) and distributed file systems designed for newly emerged dataintensive applications.  ...  The N-1 (N-to-1) parallel file write is a critical technical challenge, because it is very common for HPC applications but HDFS does not allow it.  ...  SCALER improves upon this by adding HDFS inter-block and intra-block parallel file write support.  ... 
doi:10.1109/cluster.2014.6968736 dblp:conf/cluster/YangYJS14 fatcat:h3r7eawslzc45eo5alhdevm2ea

A Survey of Semantics-Aware Performance Optimization for Data-Intensive Computing [article]

Bingbing Rao, Liqiang Wang
2021 arXiv   pre-print
This paper is aimed to provide a comprehensive understanding about current semantics-aware approaches to improve the performance of data-intensive computing.  ...  This paradigm gives rise to the term of data-intensive computing, which denotes a data parallel approach to process massive volume of data.  ...  Fault-tolerance and dynamic scalability support adding or removing computing nodes without altering the existing systems and programs, which makes it one of most widespread systems in the data-intensive  ... 
arXiv:2107.11540v1 fatcat:njo5wuctovgrti4sw6wwp6tvkq

SAGS: A SLA-Aware Green Scheduling in Heterogeneous Cloud Using Hadoop YARN

Yadaiah Balagoni, Mahatma Gandhi Institute of Technology, Ramisetty Rao, University College of Engineering, Vizianagaram
2018 International Journal of Intelligent Engineering and Systems  
Hence, this paper proposes the problem of energy-aware heterogeneous Hadoop Yarn cloud with deadline based SLA.  ...  The Apache Hadoop with cloud had become an emerging and popular service. Irrespective of its huge dominance in large scale data processing, it has challenges yet to be addressed.  ...  MapReduce was originally designed by Google [1] , to address the scalability problem in their search system.  ... 
doi:10.22266/ijies2018.1231.11 fatcat:2nqd6fdrjjeadfdtzqxejfayui

Providing Scalable Database Services on the Cloud [chapter]

Chun Chen, Gang Chen, Dawei Jiang, Beng Chin Ooi, Hoang Tam Vo, Sai Wu, Quanqing Xu
2010 Lecture Notes in Computer Science  
The storage system and the processing engine are loosely coupled, and have been designed to handle two types of workload simultaneously, namely data-intensive analytical jobs and online transactions (commonly  ...  In this paper, we present an overview of our current on-going work in developing epiC -an elastic and efficient power-aware data-intensive Cloud system.  ...  for his valuable comments and the numerous discussions during the course of the implementation of epiC.  ... 
doi:10.1007/978-3-642-17616-6_1 fatcat:vxojjhaguzbn3bdiphoqnor5pi

Petascale System Management Experiences

Narayan Desai, Rick Bradshaw, Cory Lueninghoener, Andrew Cherry, Susan Coghlan, William Scullin
2008 USENIX Large Installation Systems Administration Conference  
In particular, due to the scale of the system, we have faced a variety of issues, some surprising to us, that are not common in the commodity world.  ...  Petascale HPC systems are among the largest systems in the world. Intrepid, one such system, is a 40,000 node, 556 teraflop Blue Gene/P system that has been deployed at Argonne National Laboratory.  ...  In the last few months, we have added a dedicated DBA to the system management staff; this appears to be improving the situation.  ... 
dblp:conf/lisa/DesaiBLCCS08 fatcat:xde5xfnlunbodfgtcz26s4y6qm
« Previous Showing results 1 — 15 out of 14,853 results