IOPin: Runtime Profiling of Parallel I/O in HPC Systems

Seong Jo Kim, Seung Woo Son, Wei-keng Liao, Mahmut Kandemir, Rajeev Thakur, Alok Choudhary
2012 2012 SC Companion: High Performance Computing, Networking Storage and Analysis  
Many I/O-and data-intensive scientific applications use parallel I/O software to access files in high performance. On modern parallel machines, the I/O software consists of several layers, including high-level libraries such as Parallel netCDF and HDF, middleware such as MPI-IO, and low-level POSIX interface supported by the file systems. For the I/O software developers, ensuring data flow is important among these software layers with performance close to the hardware limits. This task requires
more » ... This task requires understanding the design of individual libraries and the characteristics of data flow among them. In this paper, we propose a dynamic instrumentation framework that can be used to understand the complex interactions across different I/O layers from applications to the underlying parallel file systems. Our preliminary experience indicates that the costs of using the proposed dynamic instrumentation is about 7% of the application execution time.
doi:10.1109/sc.companion.2012.14 dblp:conf/sc/KimSLKTC12 fatcat:ucoeziowqzhafc5fpll2bhatpy