Filters








6,960 Hits in 11.6 sec

A Segment-Level Adaptive Data Layout Scheme for Improved Load Balance in Parallel File Systems

Huaiming Song, Yanlong Yin, Xian-He Sun, Rajeev Thakur, Samuel Lang
2011 2011 11th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing  
layout scheme for parallel file systems.  ...  Keywords-parallel file system, adaptive data layout, file segment, optimal stripe size, I/O performance  ...  This research was supported in part by National Science Foundation under NSF grant CCF-0621435 and CCF-0937877, and in part by the Office of Advanced Scientific Computing Research, Office of Science, U.S  ... 
doi:10.1109/ccgrid.2011.26 dblp:conf/ccgrid/SongYSTL11 fatcat:f4xfclb7mna7xgdya3ncma6sfe

A Server-Level Adaptive Data Layout Strategy for Parallel File Systems

Huaiming Song, Hui Jin, Jun He, Xian-He Sun, Rajeev Thakur
2012 2012 IEEE 26th International Parallel and Distributed Processing Symposium Workshops & PhD Forum  
Keywords-Server-level adaptive data layout, variable stripe size, equal-depth histogram, data layout optimization, parallel file system 2012 IEEE 26th International Parallel and Distributed Processing  ...  Parallel file systems are widely used for providing a high degree of I/O parallelism to mask the gap between I/O and memory speed.  ...  In this paper, we take into consideration both service rate and bandwidth on all file servers and propose a server-level adaptive data layout strategy for parallel file systems.  ... 
doi:10.1109/ipdpsw.2012.246 dblp:conf/ipps/SongJHST12 fatcat:37msceei6vablh5kx3uqens5fa

Distributing Orthogonal Redundancy on Adaptive Disk Arrays [chapter]

J. L. Gonzalez, Toni Cortes
2008 Lecture Notes in Computer Science  
When upgrading storage systems, the key is migrating data from old storage subsystems to the new ones for achieving a data layout able to deliver high performance I/O, increased capacity and strong data  ...  This paper presents a deterministic placement approach for distributing orthogonal redundancy on distributed heterogeneous disk arrays, which is able to adapt on-line the storage system to the capacity  ...  Defining a solution at file system level : This solution can choose between using either sequential or random data placement method while using name / metadata servers for locating data [14] [15] [16  ... 
doi:10.1007/978-3-540-88871-0_64 fatcat:khda3pkq2zd5pdk2ix66ydsasy

PSA: A Performance and Space-Aware Data Layout Scheme for Hybrid Parallel File Systems

Shuibing He, Yan Liu, Xian-He Sun
2014 2014 International Workshop on Data Intensive Scalable Computing Systems  
In this paper, we propose PSA, a novel data layout scheme, which maximizes the hybrid PFSs performance by applying adaptive varied-size file stripes.  ...  We have implemented PSA within OrangeFS, a popular parallel file system in the HPC domain.  ...  Segment-level layout scheme logically divides a file into several sections such that an optimal stripe size is assigned for each section with non-uniform access patterns [10] .  ... 
doi:10.1109/discs.2014.10 dblp:conf/sc/HeLS14 fatcat:74gpqykygjcnthllhshv5yxa7q

HAS: Heterogeneity-Aware Selective Data Layout Scheme for Parallel File Systems on Hybrid Servers

Shuibing He, Xian-He Sun, Adnan Haider
2015 2015 IEEE International Parallel and Distributed Processing Symposium  
Hybrid parallel file systems (PFS), consisting of multiple HDD and SSD I/O servers, provide a promising design for data intensive applications.  ...  In this paper, we propose HAS, a novel heterogeneity-aware selective data layout scheme for hybrid PFSs.  ...  Segment-level layout scheme logically divides a file to several parts and appoints an optimal stripe size for each part [26] .  ... 
doi:10.1109/ipdps.2015.23 dblp:conf/ipps/HeSH15 fatcat:4tdc6nqjojfd5gntoyffqpkvyu

A Heterogeneity-Aware Region-Level Data Layout for Hybrid Parallel File Systems

Shuibing He, Xian-He Sun, Yang Wang, Antonis Kougkas, Adnan Haider
2015 2015 44th International Conference on Parallel Processing  
In this study, we propose a heterogeneity-aware region-level (HARL) data layout scheme to improve the data distribution of a hybrid PFS.  ...  Parallel file systems (PFS) are commonly used in high-end computing systems.  ...  Segment-level layout scheme logically divides a file to several parts and appoints an optimal stripe size for each part [10] .  ... 
doi:10.1109/icpp.2015.43 dblp:conf/icpp/HeSWKH15 fatcat:jod7lamwsnbpzo4zjwqj7had5a

Performance-Aware Data Placement in Hybrid Parallel File Systems [chapter]

Shuibing He, Xian-He Sun, Bo Feng, Kun Feng
2014 Lecture Notes in Computer Science  
Hybrid parallel file systems (PFS), which consist of both HDD and SSD servers, provide a promising solution for data-intensive applications.  ...  We have implemented PADP within OrangeFS, a widely used parallel file system in HPC domain.  ...  Segment-level placement scheme logically divides a file into several segments such that an optimal stripe size is assigned for each segment with non-uniform access patterns [6] .  ... 
doi:10.1007/978-3-319-11197-1_43 fatcat:mrgx7r5mmvg4ff3c6zypujnwpu

Boosting Application-Specific Parallel I/O Optimization Using IOSIG

Yanlong Yin, Surendra Byna, Huaiming Song, Xian-He Sun, Rajeev Thakur
2012 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (ccgrid 2012)  
Many scientific applications spend a significant portion of their execution time in accessing data from files.  ...  In this paper, we introduce a tool, called IOSIG, for providing a better understanding of parallel I/O accesses and information to be used for optimization techniques.  ...  This research was supported in part by National Science Foundation under NSF grant CCF-0621435 and CCF-0937877, and in part by the Office of Advanced Scientific Computing Research, Office of Science, U.S  ... 
doi:10.1109/ccgrid.2012.136 dblp:conf/ccgrid/YinBSST12 fatcat:phdsllr3jvasvptwkqizgbmsay

Clustered Multimedia Servers: Architectures and Storage Systems [chapter]

Guang Tan, Hai Jin, Song Wu
2003 Annual Review of Scalable Computing  
We then examine the data organization methods employed in multimedia servers, which shape the system behavior in many aspects.  ...  To reveal the essentials of multimedia service, we take a closer look at the real-time scheduling techniques for streaming data.  ...  For example, in Fig. 3 .1, 12 segments of a continuous media file are placed on the 6 nodes (labeled as S[i]) in a round robin manner and Segment 0-5 of this file can be accessed in parallel.  ... 
doi:10.1142/9789812775498_0004 fatcat:hx2w57qexvemjgxnjrrvlbti5q

Hierarchical Indexing for Out-of-Core Access to Multi-Resolution Data [chapter]

Valerio Pascucci, Randall J. Frank
2003 Mathematics and Visualization  
A key component of these schemes has become the adaptive traversal of hierarchical data-structures to build, in real time, approximate representations of the input geometry for rendering.  ...  This paper introduces a new global indexing scheme that accelerates adaptive traversal of geometric data represented with binary trees by improving the locality of hierarchical/spatial data access.  ...  As expected, the number of Data Servers has a major impact on the load balancing of the system.  ... 
doi:10.1007/978-3-642-55787-3_14 fatcat:3vbgluxt4rdr3fdelfreqm3bhu

Hibernator

Qingbo Zhu, Zhifeng Chen, Lin Tan, Yuanyuan Zhou, Kimberly Keeton, John Wilkes
2005 ACM SIGOPS Operating Systems Review  
Our solution, Hibernator, is a disk array energy management system that provides improved energy savings while meeting performance goals.  ...  Although several attempts have been made to improve disk array energy management, the existing solutions either provide little energy savings or significantly degrade performance for data center workloads  ...  Acknowledgments The authors would like to thank the shepherd, Lorenzo Alvisi, and the anonymous reviewers for their feedback.  ... 
doi:10.1145/1095809.1095828 fatcat:74wkpxpbsvfrjgetgjidgxx47q

Hibernator

Qingbo Zhu, Zhifeng Chen, Lin Tan, Yuanyuan Zhou, Kimberly Keeton, John Wilkes
2005 Proceedings of the twentieth ACM symposium on Operating systems principles - SOSP '05  
Our solution, Hibernator, is a disk array energy management system that provides improved energy savings while meeting performance goals.  ...  Although several attempts have been made to improve disk array energy management, the existing solutions either provide little energy savings or significantly degrade performance for data center workloads  ...  Acknowledgments The authors would like to thank the shepherd, Lorenzo Alvisi, and the anonymous reviewers for their feedback.  ... 
doi:10.1145/1095810.1095828 dblp:conf/sosp/ZhuCTZKW05 fatcat:nkp3atsy4be7fm2ijtxgt5gv3u

LiU: Hiding Disk Access Latency for HPC Applications with a New SSD-Enabled Data Layout

Dachuan Huang, Xuechen Zhang, Wei Shi, Mai Zheng, Song Jiang, Feng Qin
2013 2013 IEEE 21st International Symposium on Modelling, Analysis and Simulation of Computer and Telecommunication Systems  
We have implemented a prototype of LiU in the PVFS parallel file system and evaluated it with representative MPI-IO microbenchmarks, including mpi-io-test, mpi-tile-io, and ior-mpi-io, and one macro-benchmark  ...  The key technique is leveraging limited amount of SSD storage for its low-latency access, and changing data layout in a hybrid storage hierarchy with low-latency SSD at the top and high-latency hard disk  ...  ACKNOWLEDGEMENTS The authors would like to thank the anonymous reviewers for their helpful feedback.  ... 
doi:10.1109/mascots.2013.19 dblp:conf/mascots/HuangZSZJQ13 fatcat:5ilebvafubfvrehylxbhho624e

Kinesis

John MacCormick, Nicholas Murphy, Venugopalan Ramasubramanian, Udi Wieder, Junfeng Yang, Lidong Zhou
2009 ACM Transactions on Storage  
Kinesis is a novel data placement model for distributed storage systems.  ...  These design principles enable storage systems to achieve balanced utilization of storage and network resources in the presence of incremental system expansions, failures of single and shared components  ...  Morton for providing the workload traces that drove the evaluation of Kinesis. We also thank U. Erlingsson, F. McSherry, and K. Talwar for inspiring discussions on hashing and load balancing.  ... 
doi:10.1145/1480439.1480440 fatcat:q5hpe2qqtrb3vgfp4tm43vsamy

Skewed Data Distribution for Active Storage Systems on Hybrid Servers

Xiangyu Li, Shuibing He, Xianbin Xu
2016 International Journal of Grid and Distributed Computing  
We have implemented a prototype of our proposed data layout scheme in a parallel I/O system, and demonstrated its benefits with a typical data processing application.  ...  In this paper, we propose SDD, a Skewed Data Distribution scheme for hybrid active storage systems.  ...  For complex patterns, segment-level placement scheme logically divides a file into several segments such that an optimal stripe size is assigned for each segment with non-uniform access patterns [31]  ... 
doi:10.14257/ijgdc.2016.9.5.10 fatcat:vhfbeqoi3ffe7n645d4idoj4tu
« Previous Showing results 1 — 15 out of 6,960 results