Multistage switches are not crossbars: Effects of static routing in high-performance networks

Torsten Hoefler, Timo Schneider, Andrew Lumsdaine
2008 2008 IEEE International Conference on Cluster Computing  
Multistage interconnection networks based on central switches are ubiquitous in high-performance computing. Applications and communication libraries typically make use of such networks without consideration of the actual internal characteristics of the switch. However, application performance of these networks, particularly with respect to bisection bandwidth, does depend on communication paths through the switch. In this paper we discuss the limitations of the hardware (capacity-based)
more » ... on of bisection bandwidth and introduce a new metric: effective bisection bandwidth. We assess the effective bisection bandwidth of several large-scale production clusters by simulating artificial communication patterns on them. Networks with full bisection bandwidth typically provided effective bisection bandwidth in the range of 55-60%. Simulations with application-based patterns showed that the difference between effective and rated bisection bandwidth could impact overall application performance by up to 12%.
doi:10.1109/clustr.2008.4663762 dblp:conf/cluster/HoeflerSL08 fatcat:kmc2mvaw6bbtnkw7glguyhgnha