Filters








36 Hits in 4.5 sec

Unifier: Unifying cache management and communication buffer management for PVFS over intiniband

Jiesheng Wu, P. Wyckoff, D. Pandal, R. Ross
IEEE International Symposium on Cluster Computing and the Grid, 2004. CCGrid 2004.  
Unifier has three notable features. (1) Unifier integrates cache management and communication buffer management.  ...  This paper presents the design and implementation of Unifier. This component has been deployed and evaluated in a version of PVFS1 implementation over InfiniBand.  ...  Acknowledgments We are grateful to Phil Carns from Clemson University for providing us many insights into PVFS2 and many helps during our design and implementation.  ... 
doi:10.1109/ccgrid.2004.1336646 dblp:conf/ccgrid/WuWPR04 fatcat:nf3oyqlimjhonclepadrvn25oy

AHPIOS: An MPI-Based Ad Hoc Parallel I/O System

Florin Isaila, Javier Garcia Blas, Jesus Carretero, Wei-keng Liao, Alok Choudhary
2008 2008 14th IEEE International Conference on Parallel and Distributed Systems  
Additionally, the design unifies the configuration of the MPI-IO library and the AHPIOS data servers.  ...  AH-PIOS virtualizes on-demand available distributed storage resources and allows the files to be striped over several storage devices.  ...  The communication inside the file systems and in the MPICH2 was done with TCP/IP sockets over Infiniband.  ... 
doi:10.1109/icpads.2008.50 dblp:conf/icpads/IsailaBCLC08 fatcat:2jnwgugypvdp7hvoy6242ahrqm

Demotion-based exclusive caching through demote buffering

Jiesheng Wu, Pete Wyckoff, Dhabaleswar K. Panda
2003 Proceedings of the international workshop on Storage network architecture and parallel I/Os - SNAPI '03  
In this paper, we propose a DEMOTE buffering mechanism over storage networks to reduce the visible costs of DEMOTE operations and provides more flexibility for optimizations.  ...  A maximum speedup of 1.4x over the original DEMOTE approach is achieved for some workloads. 1.08-1.15x speedups are achieved for two real-life workloads.  ...  Wong for his simulator and for his answers to our questions on the DE-MOTE exclusive caching. We also want to thank anonymous reviewers for providing helpful comments on this paper.  ... 
doi:10.1145/1162618.1162627 fatcat:25cda2vtnnc3xmzhkqxa2573om

A Scalable Message Passing Interface Implementation of an Ad-Hoc Parallel I/o system

Florin Isaila, Francisco Javier Garcia Blas, Jesús Carretero, Wei-keng Liao, Alok Choudhary
2009 The international journal of high performance computing applications  
The configuration of both the MPI-IO and the storage management system is unified and allows for a tight integration of the optimizations of these layers.  ...  Our experimental section demonstrates that, with various optimizations, integrated AHPIOS offers a substantial performance benefit over the traditional MPI-IO solutions on both PVFS or Lustre parallel  ...  Department of Energy (DOE) SCIDAC-2: Scientific Data Management Center for Enabling Technologies grant DE-FC02-07ER25808, DOE FASTOS award number DE-FG02-08ER25848.  ... 
doi:10.1177/1094342009347890 fatcat:co3vcntmijewrlbrc67otrfcgi

High performance RDMA-based design of HDFS over InfiniBand

N. S. Islam, M. W. Rahman, J. Jose, R. Rajachandrasekar, H. Wang, H. Subramoni, C. Murthy, D. K. Panda
2012 2012 International Conference for High Performance Computing, Networking, Storage and Analysis  
Experimental results show that, for 5GB HDFS file writes, the new design reduces the communication time by 87% and 30% over 1Gigabit Ethernet (1GigE) and IP-over-InfiniBand (IPoIB), respectively, on QDR  ...  The existing implementation of HDFS uses Javasocket interface for communication which delivers suboptimal performance in terms of latency and throughput.  ...  We use the inhouse InfiniBand light weight communication library, Unified Communication Runtime (UCR) [10] , in our design.  ... 
doi:10.1109/sc.2012.65 dblp:conf/sc/IslamWJRWSMP12 fatcat:z3qahxdfdjcsrd73uikhlzhcya

A BeeGFS-Based Caching File System for Data-Intensive Parallel Computing [chapter]

David Abramson, Chao Jin, Justin Luong, Jake Carroll
2020 Lecture Notes in Computer Science  
The solution unifies data access for both the internal storage and external file systems using a uniform namespace.  ...  This approach has been exemplified in the design of "burst buffers", but more general caching systems have also been built.  ...  We thank HUAWEI for funding this research project. We also acknowledge the University of Queensland who provided access to FlashLite.  ... 
doi:10.1007/978-3-030-48842-0_1 fatcat:xkcvamltifhmxdisxdpxftfj54

I/O performance challenges at leadership scale

Samuel Lang, Philip Carns, Robert Latham, Robert Ross, Kevin Harms, William Allcock
2009 Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis - SC '09  
Today's top high performance computing systems run applications with hundreds of thousands of processes, contain hundreds of storage nodes, and must meet massive I/O requirements for capacity and performance  ...  We show that Intrepid's file and storage system sustain high performance under varying workloads as the applications scale with the number of processes.  ...  Future work will focus on characterizing system performance over an extended period of time.  ... 
doi:10.1145/1654059.1654100 dblp:conf/sc/LangCLRHA09 fatcat:ztzndu4sivgbvctctnbkcstnaa

High performance support of parallel virtual file system (PVFS2) over Quadrics

Weikuan Yu, Shuang Liang, Dhabaleswar K. Panda
2005 Proceedings of the 19th annual international conference on Supercomputing - ICS '05  
Quadrics QDMA and RDMA mechanisms are integrated and optimized for high performance data communication.  ...  Experimental results indicate that the performance of PVFS2, with Quadrics user-level protocols and RDMA operations, is significantly improved in terms of both data transfer and management operations.  ...  Pete Wyckoff from Ohio Supercomputing Center and Dr. Jiesheng Wu from Ask Jeeves, Inc for many technical discussions. We would like to thank members from the PVFS2 team for their technical help.  ... 
doi:10.1145/1088149.1088192 dblp:conf/ics/YuLP05 fatcat:mpv3r6hmjzd4ben2zioklh2zs4

Deep scientific computing requires deep data

W. T. C. Kramer, A. Shoshani, D. A. Agarwal, B. R. Draney, G. Jin, G. F. Butler, J. A. Hules
2004 IBM Journal of Research and Development  
The paper discusses some of the storage systems and data management methods that are needed for computing facilities to address the challenges and describes some ongoing improvements.  ...  multi-terabyte files in a timely and secure manner.  ...  A Hierarchical Resource Manager (HRM) is a TRM that has a staging disk cache for its own use. It can use the disk cache for pre-staging files for clients and for sharing files between clients.  ... 
doi:10.1147/rd.482.0209 fatcat:cqqsgvj4s5fi7kqxla7rvomujq

Cluster Computing in the Classroom and Integration With Computing Curricula 2001

A. Apon, J. Mache, R. Buyya, H. Jin
2004 IEEE Transactions on Education  
, and applications.  ...  Combining experiences in teaching cluster computing at universities in the USA and Australia, the authors present prospective topics in cluster computing and a wide variety of information sources from  ...  Networks used for interconnecting cluster 14 nodes can be local area networks such as Ethernet and Fast Ethernet; system area networks, such as Myrinet and Quadrics switches; or the InfiniBand communication  ... 
doi:10.1109/te.2004.824842 fatcat:jl5wbr7zlzddvfr54jlxylhst4

The RAMDISK storage accelerator

Tim Wickberg, Christopher Carothers
2012 Proceedings of the 2nd International Workshop on Runtime and Operating Systems for Supercomputers - ROSS '12  
LIST OF TABLES 5 Wolf von Maltzahn for their guidance and thoughful advice over the years.  ...  In this model, data is written to an accelerated disk storage layer that directly pushes data back to a larger persistent storage layer and acts as an accelerated buffer, rather than the dedicated cache  ... 
doi:10.1145/2318916.2318922 fatcat:dnl7mhmh3fe2vnvcj34saxqlqm

GraphTrek: Asynchronous Graph Traversal for Property Graph-Based Metadata Management

Dong Dai, Philip Carns, Robert B. Ross, John Jenkins, Kyle Blauer, Yong Chen
2015 2015 IEEE International Conference on Cluster Computing  
We also outline a traversal-aware query language and key optimizations (traversal-affiliate caching and execution merging) necessary for efficient performance.  ...  A property graph can be used to record the relationships between users, jobs, and data, for example, with unique annotations for each entity.  ...  In our research, we are developing a rich metadata management system based on the new concept of unifying metadata into one generic property graph [1] .  ... 
doi:10.1109/cluster.2015.48 dblp:conf/cluster/DaiCRJBC15 fatcat:27bx3br7pbejfnho5rikdr55ry

Cluster Computing White Paper [article]

Mark Baker, et. al
2001 arXiv   pre-print
The status of this paper is draft and we are at the stage of publicizing its presence and making a Request For Comments (RFC).  ...  their time and effort to put together this White Paper.  ...  References There are many different HA products available to the cluster community, each with several HA applications enabled for them in a pre-packaged form, and others being supported by various vendors  ... 
arXiv:cs/0004014v2 fatcat:cx3lbz6fcbcmrgicye7ure7u3u

Evaluating the Benefits of Key-Value Databases for Scientific Applications [chapter]

Pol Santamaria, Lena Oden, Eloy Gil, Yolanda Becerra, Raül Sirvent, Philipp Glock, Jordi Torres
2019 Lecture Notes in Computer Science  
The convergence of Big Data applications with High -Performance Computing requires new methodologies to store, manage and process large amounts of information.  ...  For example, the brain atlas of the Human Brain Project has the challenge to process large amounts of high-resolution brain images.  ...  Moreover, they were not designed for HPC and, for instance, communication is done through TCP/IP over Infiniband.  ... 
doi:10.1007/978-3-030-22734-0_30 fatcat:vjlt4pqsona4njuxdob64fgg4a

Resource Sharing for Multi-Tenant NoSQL Data Store in Cloud [article]

Jiaan Zeng
2016 arXiv   pre-print
In the shared data-parallel file system case, it has been shown that running a distributed NoSQL store over PFS for shared data across tenants is not cost effective.  ...  This dissertation targets the key-value store (KVS), a specific form of NoSQL stores, and proposes a lightweight KVS over a parallel file system to improve efficiency.  ...  She guided me into the research field of data management system and inspired my thinking; she gave me numerous opportunities to explore different directions and identify the research problems.  ... 
arXiv:1601.00738v1 fatcat:ef3aivaqafanfjamo3yahbvu64
« Previous Showing results 1 — 15 out of 36 results