On Energyaware Allocation and Execution for Batch and Interactive MapReduce

Yijun Ying, Robert Birke, Cheng Wang, Lydia Y. Chen, Gautam Natarajan
<span title="2015-06-02">2015</span> <i title="Association for Computing Machinery (ACM)"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/pkal2zplnrarzkkzugv4u5ithy" style="color: black;">Performance Evaluation Review</a> </i> &nbsp;
The energy-performance optimization of datacenters becomes ever challenging, due to heterogeneous workloads featuring different performance constraints. In addition to conventional web service, MapReduce presents another important workload class, whose performance highly depends on data availability/locality and shows different degrees of delay sensitivities, such as batch vs. interactive MapReduce. However, current energy optimization solutions are mainly designed for a subset of these
more &raquo; ... s and their key features. Here, we present an energy minimization framework, in particular, a concave minimization problem, that specifically considers time variability, data locality, and delay sensitivity of web, batch-, and interactive-MapReduce. We aim to maximize the usage of MapReduce servers by using their spare capacity to run non-MapReduce workloads, while controlling the workload delays through the execution of MapReduce tasks, in particular batch ones. We develop an optimal algorithm with complexity O(T 2 ) in case of perfect workload information, T being the length of the time horizon in number of control windows, and derive the structure of optimal policy for the case of uncertain workload information. Using extensive simulation results, we show that the proposed methodology can efficiently minimize the datacenter energy cost while fulfilling the delay constraints of workloads.
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1145/2788402.2788407">doi:10.1145/2788402.2788407</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/lptjsy3gdjaa3jt2fyvqpu7sj4">fatcat:lptjsy3gdjaa3jt2fyvqpu7sj4</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20160310092758/http://ise.tamu.edu/people/faculty/Gautam/papers/per15.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/58/ec/58ec1d3bb995a10810b394081513c5aae04f3bf2.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1145/2788402.2788407"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> acm.org </button> </a>