Tuning the performance of I/O-intensive parallel applications

Anurag Acharya, Mustafa Uysal, Robert Bennett, Assaf Mendelson, Michael Beynon, Jeff Hollingsworth, Joel Saltz, Alan Sussman
1996 Proceedings of the fourth workshop on I/O in parallel and distributed systems part of the federated computing research conference - IOPADS '96  
Getting good I O performance from parallel programs is a critical problem for many application domains. In this paper, we report our experience tuning the I O performance of four application programs from the areas of satellite-data processing and linear algebra. After tuning, three of the four applications achieve application-level I O rates of over 100 MB s on 16 processors. The total volume of I O required by the programs ranged from about 75 MB to over 200 GB. We report the lessons learned
more » ... he lessons learned in achieving high I O performance from these applications, including the need for code restructuring, local disks on every node and knowledge of future I O requests. We also report our experience on achieving high performance on peer-to-peer con gurations. Finally, w e comment on the necessity of complex I O interfaces like collective I O and strided requests to achieve high performance.
doi:10.1145/236017.236027 dblp:conf/iopads/AcharyaUBMBHSS96 fatcat:2ljwxgb2uvdhplib3wq5zqlmim