Hierarchical MapReduce Programming Model and Scheduling Algorithms

Yuan Luo, Beth Plale
2012 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (ccgrid 2012)  
We present a Hierarchical MapReduce framework that gathers computation resources from different clusters and runs MapReduce jobs across them. The applications implemented in this framework adopt the Map-Reduce-GlobalReduce model where computations are expressed as three functions: Map, Reduce, and GlobalReduce. Two scheduling algorithms are introduced: Compute Capacity Aware Scheduling for compute-intensive jobs and Data Location Aware Scheduling for data-intensive jobs. Experimental
more » ... using a molecule binding prediction tool, AutoDock, and grep demonstrate promising results for our framework.
doi:10.1109/ccgrid.2012.132 dblp:conf/ccgrid/LuoP12 fatcat:56nbhhlslbezrhgxlfzhgxv7nm