4 Hits in 4.4 sec

Efficient data restructuring and aggregation for I/O acceleration in PIDX

Sidharth Kumar, Venkatram Vishwanath, Philip Carns, Joshua A. Levine, Robert Latham, Giorgio Scorzelli, Hemanth Kolla, Ray Grout, Robert Ross, Michael E. Papka, Jacqueline Chen, Valerio Pascucci
2012 2012 International Conference for High Performance Computing, Networking, Storage and Analysis  
This work presents a new technique for parallel HZ ordering of simulation datasets that restructures simulation data into large (power of 2) blocks to facilitate efficient I/O aggregation.  ...  However, when the dimensions of the simulation data are not an even power of 2, parallel HZ ordering produces sparse memory and network access patterns that inhibit I/O performance.  ...  This algorithm extends the two-phase writing (aggregation and disk I/O) used for regular data [5] with a three-phase scheme that restructures irregular data in preparation for aggregation.  ... 
doi:10.1109/sc.2012.54 dblp:conf/sc/KumarVCLLSKGRPCP12 fatcat:uy5ntzyrjjam7mes5r7glpj5ki

A Checkpoint of Research on Parallel I/O for High-Performance Computing

Francieli Zanon Boito, Eduardo C. Inacio, Jean Luca Bez, Philippe O. A. Navaux, Mario A. R. Dantas, Yves Denneulin
2018 ACM Computing Surveys  
As the available processing power and amount of data increase, I/O remains a central issue for the scientific community.  ...  pattern extraction techniques, and performance modeling, in addition to general aspects of parallel I/O research.  ...  [47] to tune parameters of the PIDX I/O library.  ... 
doi:10.1145/3152891 fatcat:uodqvl422rbctd5ws56o4fzkmy

Scientific Applications Performance Evaluation on Burst Buffer [chapter]

George S. Markomanolis, Bilel Hadri, Rooh Khurram, Saber Feki
2017 Lecture Notes in Computer Science  
We present an in-depth analysis on how to use Burst Buffer for specific cases and how the internal MPI I/O aggregators operate according to the options that the user provides during his job submission.  ...  The storage hierarchy includes nowadays additional layers, the latest being the usage of SSD-based storage as a Burst Buffer for I/O acceleration.  ...  Initially, we have the restructuring; blocks of data are created to optimize the layout for I/O.  ... 
doi:10.1007/978-3-319-67630-2_50 fatcat:z6ph7cp4hjb5jjpcztv5g5yjli

A scalable and tunable adaptive resolution parallel i/o framework

Sidharth Kumar
Moreover, the two components of I/O, data generation from simulations (writes) and data exploration for analysis and visualization (reads), have substantially different data access requirements.  ...  To demonstrate the efficacy of the approach, a novel library (PIDX) is developed that is highly tunable and capable of adaptive-resolution parallel I/O to a multiresolution data format.  ...  Block diagram showing the three I/O phases of PIDX -restructuring, aggregation, and file I/O. 7.1.1 Three phases of I/O In the first I/O phase (data restructuring), data are redistributed from initial  ... 
doi:10.26053/0h-rvmj-htg0 fatcat:7xzngi7gjzdr7nbrqu4ajtzyom