Lessons from the Congested Clique Applied to MapReduce [article]

James W. Hegeman, Sriram V. Pemmaraju
2014 arXiv   pre-print
The main results of this paper are (I) a simulation algorithm which, under quite general constraints, transforms algorithms running on the Congested Clique into algorithms running in the MapReduce model, and (II) a distributed O(Δ)-coloring algorithm running on the Congested Clique which has an expected running time of (i) O(1) rounds, if Δ≥Θ(^4 n); and (ii) O( n) rounds otherwise. Applying the simulation theorem to the Congested-Clique O(Δ)-coloring algorithm yields an O(1)-round O(Δ)-coloring
more » ... algorithm in the MapReduce model. Our simulation algorithm illustrates a natural correspondence between per-node bandwidth in the Congested Clique model and memory per machine in the MapReduce model. In the Congested Clique (and more generally, any network in the CONGEST model), the major impediment to constructing fast algorithms is the O( n) restriction on message sizes. Similarly, in the MapReduce model, the combined restrictions on memory per machine and total system memory have a dominant effect on algorithm design. In showing a fairly general simulation algorithm, we highlight the similarities and differences between these models.
arXiv:1405.4356v2 fatcat:vabcraojond3fe5454fv6c3ve4