Topology-aware resource management for HPC applications

Yiannis Georgiou, Emmanuel Jeannot, Guillaume Mercier, Adèle Villiermet
2017 Proceedings of the 18th International Conference on Distributed Computing and Networking - ICDCN '17  
The Resource and Job Management System (RJMS) is a crucial system software part of the HPC stack. It is responsible for eciently delivering computing power to applications in supercomputing environments. Its main intelligence relies on resource selection techniques to nd the most adapted resources to schedule the users' jobs. Improper resource selection operations may lead to poor performance executions and global system utilization along with increase of system fragmentation and jobs
more » ... . These phenomenas play a role in the increase of the platforms' total cost of ownership and should be minimized. This paper introduces a new topology-aware resource selection algorithm to determine the best choice among the available nodes of the platform based upon their position within the network and taking into account the applications communication matrix. To validate our approach, we integrated this algorithm as a plugin for Slurm, a popular and widespread HPC resource and job management system (RJMS). We validated our plugin with dierent optimization schemes by comparing with the default Slurm algorithm using both emulation of a large-scale platform, and by carrying out experiments in a real cluster.
doi:10.1145/3007748.3007768 fatcat:sflvwmx525gubn6svjanowe6fe