23 Hits in 8.0 sec

Improving I/O Performance for Exascale Applications through Online Data Layout Reorganization [article]

Lipeng Wan, Axel Huebl, Junmin Gu, Franz Poeschel, Ana Gainaru, Ruonan Wang, Jieyang Chen, Xin Liang, Dmitry Ganyushin, Todd Munson, Ian Foster, Jean-Luc Vay (+3 others)
2021 arXiv   pre-print
We review these I/O challenges and introduce two online data layout reorganization approaches for achieving good tradeoffs between read and write performance.  ...  We show that by understanding application I/O patterns and carefully designing data layouts we can increase read performance by more than 80%.  ...  Performance results for this method, using realistically complex application I/O patterns, are readily applicable without any change to the application code.  ... 
arXiv:2107.07108v1 fatcat:6wuqtjetbrd7pksrnbpbxlslue

A Checkpoint of Research on Parallel I/O for High-Performance Computing

Francieli Zanon Boito, Eduardo C. Inacio, Jean Luca Bez, Philippe O. A. Navaux, Mario A. R. Dantas, Yves Denneulin
2018 ACM Computing Surveys  
As the available processing power and amount of data increase, I/O remains a central issue for the scientific community.  ...  We present a comprehensive survey on parallel I/O in the high performance computing (HPC) context.  ...  Acknowledgments This research has been partially supported by CAPES-BRAZIL through the project PVEA117-2013.  ... 
doi:10.1145/3152891 fatcat:uodqvl422rbctd5ws56o4fzkmy

Adaptively Periodic I/O Scheduling for Concurrent HPC Applications

Benbo Zha, Hong Shen
2022 Electronics  
Currently the existing heuristic online and periodic offline I/O scheduling methods for traditional HPC applications with a fixed I/O periodicity are not suitable for the applications with stochastically  ...  With the convergence of big data and HPC (high-performance computing), various machine learning applications and traditional large-scale simulations with a stochastically iterative I/O periodicity are  ...  In reference [31] , the proposed IOrchestrator reorganizes the I/O requests by considering the data spatial locality.  ... 
doi:10.3390/electronics11091318 fatcat:f67wcn7ts5eq5etvlpj2dpk5iy

IEEE Special Issue on Innovative R&D Toward the Exascale Era

Sadaf R. Alam, Lois Curfman McInnes, Kengo Nakajima
2022 IEEE Transactions on Parallel and Distributed Systems  
Papers in this special issue explore diverse topics in research encompassing parallel, distributed, and heterogeneous systems for exascale, including advances in applications, programming environments,  ...  Exascale computing is defined as the capability to perform 10 18 operations per second.  ...  We thank all members of the review committee for their hard work and insightful feedback on submissions. We also thank the community for interest in this special issue.  ... 
doi:10.1109/tpds.2021.3109651 fatcat:sh5naep5qzamvifo5jmgggauni

On Processing Extreme Data

Dana Petcu, Gabriel Iuhasz, Daniel Pop, Domenico Talia, Jesus Carretero, Radu Prodan, Thomas Fahringer, Ivan Grasso, Ramon Doallo, Maria J. Martin, Basilio B. Fraguela, Roman Trobec (+11 others)
2016 Scalable Computing : Practice and Experience  
metadata service for exascale I/O. 2. 3 . 3 Advancing the Balance of Compute, I/O and Storage Performance.  ...  Storage structures for exascale I/O.  ... 
doi:10.12694/scpe.v16i4.1134 fatcat:yibmtpz5szgudgocgoojodhjc4

Collective input/output under memory constraints

Yin Lu, Yong Chen, Yu Zhuang, Jialin Liu, Rajeev Thakur
2014 The international journal of high performance computing applications  
The evaluation results demonstrate that this strategy holds promise in mitigating the memory pressure, alleviating the contention for memory bandwidth, and improving the I/O performance for projected extreme-scale  ...  Compared with current high-performance computing (HPC) systems, exascale systems are expected to have much less memory per node, which can significantly reduce necessary collective input/output (I/O) performance  ...  Acknowledgements We acknowledge the High Performance Computing Center (HPCC) at Texas Tech University at Lubbock for providing HPC resources that have contributed to the research results reported in this  ... 
doi:10.1177/1094342014561696 fatcat:hjccbkbxozeadaanwkz6vvzwqe

Scalable in situ scientific data encoding for analytical query processing

Sriram Lakshminarasimhan, David A. Boyuka, Saurabh V. Pendse, Xiaocheng Zou, John Jenkins, Venkatram Vishwanath, Michael E. Papka, Nagiza F. Samatova
2013 Proceedings of the 22nd international symposium on High-performance parallel and distributed computing  
to write the raw data with MPI collective I/O.  ...  With the onset of exascale computing, the increasing gap between compute performance and I/O bandwidth has rendered the traditional method of post-simulation processing a tedious process.  ...  ACKNOWLEDGMENT We would like to thank the FLASH Center for Computational Science at the University of Chicago for providing access to the FLASH simulation code and both the FLASH and S3D teams for providing  ... 
doi:10.1145/2462902.2465527 fatcat:pdrsmcgwsnfo7otplogzif5ae4

The International Exascale Software Project roadmap

Jack Dongarra, Pete Beckman, Terry Moore, Patrick Aerts, Giovanni Aloisio, Jean-Claude Andre, David Barkai, Jean-Yves Berthou, Taisuke Boku, Bertrand Braunschweig, Franck Cappello, Barbara Chapman (+53 others)
2011 The international journal of high performance computing applications  
In developing a plan for producing a new software infrastructure capable of supporting exascale applications, we charted a path that moves through the following sequence of objectives: 1.  ...  Over the last twenty years, the open source community has provided more and more software on which the world's High Performance Computing (HPC) systems depend for performance and productivity.  ...  For example, all areas need some I/O, but the ones checked were deemed to need considerable I/O, based on the problems that exist today.  ... 
doi:10.1177/1094342010391989 fatcat:twdszcjfxraijpsdcdacvpp6vm

Software challenges in extreme scale systems

Vivek Sarkar, William Harrod, Allan E Snavely
2009 Journal of Physics, Conference Series  
Carlson is a member of the research staff at the IDA Center for Computing Sciences where, since 1990, his focus has been on applications and system tools for large-scale parallel and distributed computers  ...  a range of real system applications, from highly scalable deep space exploration to trans-petaflops level supercomputing.  ...  Only a technology shift that reduces latency (increases random I/O access speed) can improve overall performance when there is significant random I/O. • We estimate that, taken as a whole, data-intensive  ... 
doi:10.1088/1742-6596/180/1/012045 fatcat:iukutry2dvbitfdh6ng7kgz564

The Journal of High-Performance Storage, Issue 2

Julian Kunkel, Jean-Thomas Acquaviva, Suren Byna, Adrian Jackson, Ivo Jimenez, Anthony Kougkas, Jay Lofstead, Glenn K. Lockwood, Carlos Maltzahn, George S. Markomanolis, Lingfang Zeng
2021 Zenodo  
This is issue 2 of the journal of High-Performance Storage.  ...  We thank the reviewers Lingfang Zeng, Suren Byna, Anthony Kougkas, and George Markomanolis for their constructive feedback. Acknowledgment We thank the reviewers for their comments and suggestions.  ...  Acknowledgment This research used computational resources of the K computer provided by the RIKEN Center for Computational Science.  ... 
doi:10.5281/zenodo.5336896 fatcat:y5nu222nlfgzjgwzz2qpergxcy

A Survey and Classification of Software-Defined Storage Systems

Ricardo Macedo, João Paulo, José Pereira, Alysson Bessani
2020 ACM Computing Surveys  
As a result, these homogeneous setups have lead to applications running on a general-purpose I/O stack, competing for the system resources in a non-optimal fashion, and incapable of performing I/O differentiation  ...  ., caching, queueing, data management) with strict interfaces and isolated procedures to employ over I/O requests, leveraging a complex, limited, and coarse-grained treatment of the I/O flow.  ...  Clarisse [40] is a SDS system designed to improve the I/O stack performance of complex largescale HPC infrastructures.  ... 
doi:10.1145/3385896 fatcat:f4v7bgxwfbaxpfckrei4qkttby

A scalable and tunable adaptive resolution parallel i/o framework

Sidharth Kumar
This is achieved by developing technology for highly scalable and tunable parallel I/O, applicable to both traditional parallel data formats and multiresolution data formats, which are directly appropriate  ...  Adaptive resolution storage and I/O, which allows subsets of a simulation to be accessed at varying spatial resolutions, can yield significant improvements to both the storage performance and I/O time.  ...  As we move towards exascale, I/O will pose even tougher challenges.  ... 
doi:10.26053/0h-rvmj-htg0 fatcat:7xzngi7gjzdr7nbrqu4ajtzyom

New Storage Layout For Earth System Data (D4.2)

Jakob Luettgau, Julian Kunkel, Bryan Lawrence, Jens Jensen, GIuseppe Congiu, Huang Hua, Paola Nassisi
2017 Zenodo  
In the ESiWACE project, we develop a novel I/O middleware targeting, but not limited to, earth system data.  ...  Existing storage environments face usability and performance challenges for both domain scientists and the data centers supporting the scientists.  ...  I/O requests arrive through the ESDM API. The layout component provides a fragmentation based on site configuration and performance model.  ... 
doi:10.5281/zenodo.1228763 fatcat:e57r7wzxzzfc5nykde4yntk7ny

New Storage Layout for Earth System Data (D4.2)

Jakob Luettgau, Julian Kunkel, Bryan Lawrence, Jens Jensen, GIuseppe Congiu, Huang Hua, Paola Nassisi
2017 Zenodo  
In the ESiWACE project, we develop a novel I/O middleware targeting, but not limited to, earth system data.  ...  Existing storage environments face usability and performance challenges for both domain scientists and the data centers supporting the scientists.  ...  I/O requests arrive through the ESDM API. The layout component provides a fragmentation based on site configuration and performance model.  ... 
doi:10.5281/zenodo.2573896 fatcat:q67wragqkreijczychggxgkd6m

Autotuning in High-Performance Computing Applications

Prasanna Balaprakash, Jack Dongarra, Todd Gamblin, Mary Hall, Jeffrey K. Hollingsworth, Boyana Norris, Richard Vuduc
2018 Proceedings of the IEEE  
Autotuning has the potential to dramatically improve the performance portability of petascale and exascale applications.  ...  To date, autotuning has been used primarily in high-performance applications through tunable libraries or previously tuned application code that is integrated directly into the application.  ...  Data transformations may also be applied to reorganize the data layout or copy it to other memory structures.  ... 
doi:10.1109/jproc.2018.2841200 fatcat:576fhps52banrfowi6ykk7za6a
« Previous Showing results 1 — 15 out of 23 results