Compiler-directed I/O optimization

M. Kandemir, A. Choudhary
2002 Proceedings 16th International Parallel and Distributed Processing Symposium  
Despite continued innovations in design of I/O systems, I/O performance has not kept pace with the progress in processor and communication technology. This paper addresses this I/O problem from a compiler's perspective, and presents an I/O optimization strategy based on access pattern and storage form (file layout) detection. The objective of our optimization strategy is to determine storage forms for array-based data sets taking into account future use of data (future access patterns). To
more » ... e this problem, we present a three-step strategy: (i) determining all I/O access patterns to the array, and among them, selecting the most dominant (i.e., the most beneficial) access pattern; (ii) determining the most suitable storage form for the array taking into account the most dominant access pattern detected in the previous step; and (iii) optimizing the non-dominant access patterns using collective I/O, an optimization that allows each processor to do I/O on behalf of others if doing so improves overall performance.
doi:10.1109/ipdps.2002.1015492 dblp:conf/ipps/KandemirC02 fatcat:5jppwhstfbh3hopzoweibfougm