Optimization of applications with non-blocking neighborhood collectives via multisends on the Blue Gene/P supercomputer

Sameer Kumar, Philip Heidelberger, Dong Chen, Michael Hines
2010 2010 IEEE International Symposium on Parallel & Distributed Processing (IPDPS)  
We explore the multisend interface as a data mover interface to optimize applications with neighborhood collective communication operations. One of the limitations of the current MPI 2.1 standard is that the vector collective calls require counts and displacements (zero and nonzero bytes) to be specified for all the processors in the communicator. Further, all the collective calls in MPI 2.1 are blocking and do not permit overlap of communication with computation. We present the record replay
more » ... rsistent optimization to the multisend interface that minimizes the processor overhead of initiating the collective. We present four different case studies with the multisend API on Blue Gene/P (i) 3D-FFT, (ii) 4D nearest neighbor exchange as used in Quantum Chromodynamics, (iii) NAMD and (iv) neural network simulator NEURON. Performance results show 1.9x speedup with 32 3 3D-FFTs, 1.9x speedup for 4D nearest neighbor exchange with the 2 4 problem, 1.6x speedup in NAMD and almost 3x speedup in NEURON with 256K cells and 1k connections/cell.
doi:10.1109/ipdps.2010.5470407 pmid:21666880 pmcid:PMC3111918 dblp:conf/ipps/KumarHCH10 fatcat:ud3bdkbo6zf4pix4cegwnx6x3m