Enhancing MapReduce Fault Recovery Through Binocular Speculation [article]

Huansong Fu, Yue Zhu, Amit Kumar Nath, Md. Muhib Khan, Weikuan Yu
<span title="2019-01-23">2019</span> <i > arXiv </i> &nbsp; <span class="release-stage" >pre-print</span>
MapReduce speculation plays an important role in finding potential task stragglers and failures. But a tacit dichotomy exists in MapReduce due to its inherent two-phase (map and reduce) management scheme in which map tasks and reduce tasks have distinctly different execution behaviors, yet reduce tasks are dependent on the results of map tasks. We reveal that speculation policies for fault handling in MapReduce do not recognize this dichotomy between map and reduce tasks, which leads to an
more &raquo; ... of speculation myopia for MapReduce fault recovery. These issues cause significant performance degradation upon network and node failures. To address the speculation myopia caused by MapReduce dichotomy, we introduce a new scheme called binocular speculation to help MapReduce increase its assessment scope for speculation. As part of the scheme, we also design three component techniques including neighborhood glance, collective speculation and speculative rollback. Our evaluation shows that, with these techniques, binocular speculation can increase the coordination of map and reduce phases, and enhance the efficiency of MapReduce fault recovery.
<span class="external-identifiers"> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/1901.07715v1">arXiv:1901.07715v1</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/bwchplbyzncgxhpn773rfhkfsq">fatcat:bwchplbyzncgxhpn773rfhkfsq</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20200824024551/https://arxiv.org/pdf/1901.07715v1.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/15/24/152455d781a109a03a40b46c788e602233d0adba.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/1901.07715v1" title="arxiv.org access"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> arxiv.org </button> </a>