A taxonomy of market-based resource management systems for utility-driven cluster computing

Chee Shin Yeo, Rajkumar Buyya
2006 Software, Practice & Experience  
In utility-driven cluster computing, cluster systems need to know the specific needs of different users so as to allocate resources according to their needs. They are also vital in supporting service-oriented Grid computing that harness resources distributed worldwide based on users' objectives. Market-based resource management systems make use of real-world market concepts and behavior to assign resources to users. This paper outlines a taxonomy that describes how market-based resource
more » ... nt systems can support utility-driven cluster computing. The taxonomy is used to survey existing market-based resource management systems to better understand how they can be utilized. I. INTRODUCTION The next-generation scientific research involves solving Grand Challenge Applications (GCAs) that demand ever increasing amount of computing power. Recently, a new type of High Performance Computing (HPC) paradigm called cluster computing has become a more viable choice for executing these GCAs since cluster systems are able to offer equally high performance with a lower price compared to traditional supercomputing systems. A cluster system comprises of independent machines that are connected by high-speed networks and uses middlewares that create an illusion of single system [4] and hide the complexities of underlying cluster architecture from the users. For example, the cluster Resource Management System (RMS) is a middleware that manages the resources and seamless execution of jobs in a cluster of computers. Existing cluster RMSs still adopt system-centric resource allocation approaches that maximizes overall job performance and system usage. These system-centric approaches assume that all job requests are of equal importance and thus neglect actual levels of service required by different users. Market-based RMSs [5][6][7][8] [9] have a greater emphasis on user QoS requirements as opposed to traditional RMSs that focus on maximizing system usage. Market concepts can be used to prioritize competing jobs and assign resources to jobs according to users' valuation for QoS requirements and cluster resources. Market-based cluster RMSs need to support three requirements in order to enable utility-driven cluster computing [7]: (i) provide means for users to specify their Quality of Service (QoS) needs and valuations, (ii) utilize policies to translate the valuations into resource allocations, and (iii) support mechanisms to enforce the resource allocations in order to achieve each individual user's perceived value or utility. The first requirement allows the market-based cluster RMS to be aware of usercentric service requirements so that competing service requests can be prioritized more accurately. The second requirement then determines how the cluster RMS can allocate resources appropriately and effectively to different requests by considering
doi:10.1002/spe.725 fatcat:6jqpwyqfdva33ffhkfwwu6gafa