9,991 Hits in 5.2 sec

Improving I/O Performance for Exascale Applications through Online Data Layout Reorganization [article]

Lipeng Wan, Axel Huebl, Junmin Gu, Franz Poeschel, Ana Gainaru, Ruonan Wang, Jieyang Chen, Xin Liang, Dmitry Ganyushin, Todd Munson, Ian Foster, Jean-Luc Vay (+3 others)
2021 arXiv   pre-print
We review these I/O challenges and introduce two online data layout reorganization approaches for achieving good tradeoffs between read and write performance.  ...  Exascale Computing Project (ECP) to run on imminent Exascale computers will generate scientific results with unprecedented fidelity and record turn-around time.  ...  REORGANIZATION OF DATA LAYOUT Although the data blocks clustering and merging approach proposed in Section 4 improves the read performance compared to only enabling the chunking and sub-filing strategies  ... 
arXiv:2107.07108v1 fatcat:6wuqtjetbrd7pksrnbpbxlslue

Online reorganization of databases

Gary H. Sockut, Balakrishna R. Iyer
2009 ACM Computing Surveys  
to newly allocated storage (as opposed to reorganizing in place), use of differential files, references to data that has moved, performance, and activation of reorganization.  ...  in a logstructured file system.  ...  ACKNOWLEDGMENTS A suggestion from Robert Goldberg led to the first author's early work in online reorganization. We thank these people for discussing issues in reorganization: Kiran Achyutuni, S.  ... 
doi:10.1145/1541880.1541881 fatcat:nbrjxa6h6fberhpb2blzmqoio4

A Framework to improve the Web Performance using Reorganization, Optimized Prediction and Prefetching

2019 International journal of recent technology and engineering  
This paperelucidates the combination of many ideas to improve web performance and given as a framework.  ...  in webserver and finally improvements in a proxy cache at the time of accessing dynamic content.  ...  User visited Pages are recorded in the web server log file which will be the main source of information to create a rich and good adaptive website.  ... 
doi:10.35940/ijrte.c6332.098319 fatcat:g4q7wlyi6fhzldbq3otbpsmk5a

Adaptive prefetching and storage reorganization in a log-structured storage system

Chye Lin Chee, Hongjun Lu, Hong Tang, C.V. Ramamoorthy
1998 IEEE Transactions on Knowledge and Data Engineering  
The storage system can serve as a testbed for a variety of statistics analysis and clustering mechanisms.  ...  Performance results from our prototype show potential response time speedups of up to 83 percent over the basic log-structured file system in the best case, using a combination of storage reorganization  ...  Global Statistics Global statistics are characteristics common to all blocks, and determine factors such as the time that disk reorganization should take place, frequently accessed blocks, and data block  ... 
doi:10.1109/69.729739 fatcat:aq3g6c7ewfcircsvejg53ht3mu

Expediting scientific data analysis with reorganization of data

Bin Dong, Surendra Byna, Kesheng Wu
2013 2013 IEEE International Conference on Cluster Computing (CLUSTER)  
SDS reorganizes data to match the read patterns of analysis tasks and enables transparent data reads from the reorganized data.  ...  We implemented a HDF5 Virtual Object Layer (VOL) plugin to redirect the HDF5 dataset read calls to the reorganized data.  ...  To improve the performance of this read operation, an approach is to sort the data in a data set and then store the sorted data in a new file.  ... 
doi:10.1109/cluster.2013.6702675 dblp:conf/cluster/DongBW13 fatcat:or2k47ihr5bphm7itimzs3rtuq

Unified and efficient HEC storage system with a working-set based reorganization scheme

Junjie Chen, Yong Chen
2013 2013 IEEE International Conference on Cluster Computing (CLUSTER)  
In this paper, we introduce our initial study of a novel Working-Set based Reorganization Scheme (WS-ROS), to manage and leverage the merits of both SSDs and HDDs and to provide a highly efficient storage  ...  SSDs and HDDs have complement characteristics in nature and there is a desire to combine and unify them to best serve HEC workloads.  ...  ACKNOWLEDGMENT This research is sponsored in part by the Texas Tech University startup grant and the National Science Foundation under grant CNS-1162488.  ... 
doi:10.1109/cluster.2013.6702620 dblp:conf/cluster/ChenC13 fatcat:giufyi4sb5euzpptze5e76q6zi

CCAM: a connectivity-clustered access method for networks and network computations

S. Shekhar, Duen-Ren Liu
1997 IEEE Transactions on Knowledge and Data Engineering  
The nodes of the network are assigned to disk pages via a graph partitioning approach to maximize the WCRR.  ...  CCAM supports the operations of insert, delete, create, and nd as well as the new operations, get-A-successor and get-successors, which retrieve one or all successors of a node to facilitate aggregate  ...  We would like to thank Dr. H.V. Jagadish (AT&T Bell Labs) and Prof. K. Hua (University of Florida) for helping with the survey and focus of this research. We would also like to thank Prof. C.K.  ... 
doi:10.1109/69.567054 fatcat:fdfevm3fzrbbtd3runl24kf5ne

The automatic improvement of locality in storage systems

Windsor W. Hsu, Alan Jay Smith, Honesty C. Young
2005 ACM Transactions on Computer Systems  
powerful processors and large memories in storage systems have ample capacity to reorganize the data layout and redirect the accesses so as to take advantage of rapid sequential data transfer.  ...  Using trace-driven simulation with a large set of real workloads, we demonstrate that ALIS considerably outperforms prior techniques, improving the average read performance by up to 50% for server workloads  ...  In addition, the authors are grateful to John Palmer and Jai Menon for helpful comments on versions of this paper.  ... 
doi:10.1145/1113574.1113577 fatcat:jcmhhs2q2jci7nxhpildq652mi

The Power and Challenges of Transformative I/O

Adam Manzanares, John Bent, Meghan Wingate, Garth Gibson
2012 2012 IEEE International Conference on Cluster Computing  
There are at least three possible ways to address this challenge: modification of the real-world workloads, modification of the underlying parallel file systems, or reorganization of the real-world workloads  ...  In this paper, we demonstrate that transformative middleware is applicable across a large set of high performance computing workloads and is portable across the three major parallel file systems in use  ...  In order to maintain the users logical view of the file, PLFS also appends a record of each write to a unique index file for each process.  ... 
doi:10.1109/cluster.2012.86 dblp:conf/cluster/ManzanaresBWG12 fatcat:vy6ilpma6jantiu6dogargztui

An interactive DSS tool for physical database design

Prashant C. Palvia
1991 Information Sciences  
These works include: index selection [1, 14, 17] , file structuring and models of file organization [18, 30] , and record segmentation and structuring [15, 17, 23, 24] .  ...  Further, the database designer may want to experiment with design preferences and features not considered by the mathematical optimization approaches.  ...  Create two record types X and Y with X having pointers to Y. b. Create two record types X and Y with Y having pointers to X. c. Create two record types X and Y with both pointing to each other. d.  ... 
doi:10.1016/0020-0255(91)90053-w fatcat:k5lgdlbxvzevrm3lycdw6nian4

Pattern-Direct and Layout-Aware Replication Scheme for Parallel I/O Systems

Yanlong Yin, Jibing Li, Jun He, Xian-He Sun, Rajeev Thakur
2013 2013 IEEE 27th International Symposium on Parallel and Distributed Processing  
A runtime system is designed and developed to integrate the PDLA replication scheme and existing parallel I/O system; a prototype of PDLA is implemented under the MPICH2 and PVFS2 environments.  ...  The basic idea of PDLA is replicating identified data access pattern, and saving these reorganized replications with optimized data layouts based on access cost analysis.  ...  To some extent, one trace file is a list of file operation records, and each record contains an operation's data access information.  ... 
doi:10.1109/ipdps.2013.114 dblp:conf/ipps/YinLHST13 fatcat:xrkrzb2tdjd4dfwcf5edzwij2e

Automation and technical services organization

Rosann Bazirjian
1993 Library Acquisitions: Practice & Theory  
These files retained order information, contained bibliographic holdings and maintained auditing and accounting records.  ...  As we automate, files begin to disappear, and as they disappear, so do the traditional organizational structures with which we are familiar.  ...  The eight OCLC terminals grew to ten and were placed into three smaller clusters rather than one. In addition, every staff member now has a terminal at his/her desk to access NOTIS.  ... 
doi:10.1016/0364-6408(93)90033-3 fatcat:pc7rkovjwvbirdm754ep63rzva

Creative Combination of Legacy System and MapReduce in Cloud Migration

Junfeng Zhao, Wenmeng Wang
2019 International Journal of Performability Engineering  
Therefore, how to creatively combine parallelizable legacy code and the MapReduce model to enable legacy code to be accurately mapped into the MapReduce model is a challenging issue.  ...  We use the first type of creative computing to propose an approach for legacy code refactoring.  ...  Data Reorganization This type of data reorganization pays attention to reorganizing the disordered data in the original data set into relatively ordered and well-organized data according to a certain standard  ... 
doi:10.23940/ijpe.19.02.p22.579590 fatcat:ho6bw4hj4bfvxdqi7qoimhznfq

Spatially clustered join on heterogeneous scientific data sets

Bin Dong, Surendra Byna, Kesheng Wu
2015 2015 IEEE International Conference on Big Data (Big Data)  
known as Multi-Dimensional Binning (MDBin), and a join processing algorithm known as Spatially Clustered Join (SCJoin).  ...  Together, these techniques allow scientific data files to be used for query processing with less I/O cost and fast query response time without the extra cost to perform file format conversion and data  ...  Often it is also beneficial to reorder the data records following the index structure using a strategy known as clustered indexes [27] .  ... 
doi:10.1109/bigdata.2015.7363778 dblp:conf/bigdataconf/DongBW15 fatcat:v5xdukg2mfb4hkafsn5winjnq4

An Heighten PSO-K-harmonic Mean Based Pattern Recognition in User Navigation

R. Gobinath, M. Hemalatha
2014 Research Journal of Applied Sciences Engineering and Technology  
The approaches followed are to separate the users and sessions from the web log files and acquiring the necessary patterns for web personalization.  ...  This approach mines the web log files which are resultant from the web users while interacting with web pages for a particular period of web sessions.  ...  clustering algorithm for reorganizing websites by pages index.  ... 
doi:10.19026/rjaset.7.421 fatcat:znts3hify5hurlakz3ombzfcyy
« Previous Showing results 1 — 15 out of 9,991 results