Lessons from the Congested Clique applied to MapReduce

James W. Hegeman, Sriram V. Pemmaraju
<span title="">2015</span> <i title="Elsevier BV"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/elaf5sq7lfdxfdejhkqbtz6qoq" style="color: black;">Theoretical Computer Science</a> </i> &nbsp;
The main result of this paper is a simulation algorithm which, under quite general constraints, transforms algorithms running in the Congested Clique model into algorithms running in the MapReduce model. As a case study of applying this simulation algorithm, we first present a distributed O(∆)-coloring algorithm running on the Congested Clique in O(1) rounds, if ∆ ≥ log 2 n, and O(log log log n) rounds otherwise. Applying the simulation theorem to this Congested Clique O(∆)-coloring algorithm
more &raquo; ... elds an O(1)-round O(∆)-coloring algorithm in the MapReduce model. We apply our simulation algorithm to other Congested Clique algorithms including the 2-ruling set and metric facility location algorithms of Berns et al. (ICALP 2012). Our simulation algorithm illustrates a natural correspondence between per-node bandwidth in the Congested Clique model and memory per machine in the MapReduce model. In the Congested Clique (and more generally, any network in the CON GEST model), the major impediment to constructing fast algorithms is the O(log n) restriction on message sizes. Similarly, in the MapReduce model, the combined restrictions on memory per machine and total system memory have a dominant effect on algorithm design. In showing a fairly general simulation algorithm, we highlight the similarities and differences between these models. in the Congested Clique setting would imply lower bounds in circuit complexity, resolving long-standing open questions. While Lotker et al. [27] mention overlay networks as a possible practical application of distributed computation on a Congested Clique, as of now, research on this model is largely driven by a theoretical interest in exploring the limits imposed by congestion. MapReduce [7] is a tremendously popular parallel-programming framework that has become the tool of choice for large-scale data analytics at many companies such as Amazon, Facebook, Google, Yahoo!, etc., as well as at many universities. While the actual time-efficiency of a particular MapReduce-like implementation will depend on many low-level technical details, Karloff et al. [20] have attempted to formalize key constraints of this framework to propose a MapReduce model and an associated MapReduce complexity class (MRC). Informally speaking, a problem belongs to MRC if it can be solved in the MapReduce framework using: (i) a number of machines that is substantially sublinear in the input size, i.e., O(n 1− ) for constant > 0, (ii) memory per machine that is substantially sublinear in the input size, (iii) O(poly(log n)) number of map-shuffle-reduce rounds, and (iv) polynomial-time local computation at each machine in each round. Specifically, a problem is said to be in MRC i if it can be solved in O(log i n) map-shuffle-reduce rounds, while maintaining the other constraints mentioned above. Karloff et al. [20] show that minimum spanning tree (MST) is in MRC 0 (i.e., MST requires O(1) map-shuffle-reduce rounds) on non-sparse instances. Following up on this, Lattanzi et al. [23] show that other problems such as maximal matching (with which the distributed computing community is very familiar) are also in MRC 0 (again, for non-sparse instances). We give a more-detailed description of the MapReduce model in Section 1.1. The volume of communication that occurs in a Shuffle step can be quite substantial and provides a strong incentive to design algorithms in the MapReduce framework that use very few map-shuffle-reduce steps. As motivation for their approach (which they call filtering) to designing MapReduce algorithms, Lattanzi et al. [23] mention that past attempts to "shoehorn message-passing style algorithms into the framework" have led to inefficient algorithms. While this may be true for distributed message-passing algorithms in general, we show in this paper that algorithms designed in the Congested Clique model provide many lessons on how to design algorithms in the MapReduce model. We illustrate this by first designing an expected-O(1)-round algorithm for computing a O(∆)-coloring for a given n-node graph with maximum degree ∆ ≥ log 2 n in the Congested Clique model. We then simulate this coloring algorithm in the MapReduce model and obtain a corresponding algorithm that uses a constant number of map-shuffle-reduce rounds to compute an O(∆)coloring of the given graph. While both of these results are new, what we wish to emphasize in this paper is the simulation of Congested Clique algorithms in the MapReduce model. Our simulation can also be used to obtain efficient MapReduce-model algorithms for other problems such as 2-ruling sets [3] for which an expected-O(log log n)-round algorithm on a Congested Clique was recently developed. Models The Congested Clique Model. The Congested Clique is a variation on the CON GEST model. The underlying communication network is a size-n clique, i.e., every pair of nodes can directly communicate with each other. Computation proceeds in synchronous rounds and in each round a node (i) receives all messages sent to it in the previous round; (ii) performs unlimited local computation; and then (iii) sends a, possibly distinct, message of size O(log n) to each other node in the network. We assume that nodes have distinct IDs that can each be represented in O(log n) bits. We call this the Congested Clique model. Our focus in this paper is graph problems and we assume that the input is a graph G that is a spanning subgraph of the communication network. Initially, each node in the network knows who its neighbors are in G. Thus knowledge of G is distributed among the nodes of the network, with each node having a particular local view of G. Note that G can be quite dense (e.g., have Ω(n 2 ) edges) and therefore any reasonably fast algorithm for the problem will have to be "truly" distributed in the sense that it cannot simply rely on shipping off the problem description to a single node for local computation. The MapReduce Model. Our description of the MapReduce model borrows heavily from the work of Karloff et al. [20] and Lattanzi et al. [23]. Introduced by Karloff et al. [20], the MapReduce model is an abstraction 1. Key-sizes and value-sizes are restricted to a Θ(1) multiple of the word size of the system. Because of this restriction, a mapper takes as input only a constant number of words.
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1016/j.tcs.2015.09.029">doi:10.1016/j.tcs.2015.09.029</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/ilijmw3ebzf35icintfyarstfu">fatcat:ilijmw3ebzf35icintfyarstfu</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20180719090349/https://manuscript.elsevier.com/S0304397515008695/pdf/S0304397515008695.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/a4/7a/a47a0539f11f1071bde0dfcaf49ee2c51924dbfd.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1016/j.tcs.2015.09.029"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> elsevier.com </button> </a>