A vibrant data placement approach for map reduce in diverse environments

J Sujatha, K Meena
2018 International Journal of Engineering & Technology  
Map reduce assumes that the computing capacity is same for each node in a cluster. Each node is assigned to the same load in homogeneous environment, hence it fully use the resources in the cluster. In such a cluster, there is likely to be various specifications of PCs or servers, which causes the abilities of the nodes to differ. If such a heterogeneous environment still uses the original Hadoop strategy that distributes data blocks into each node equally and the load is also evenly
more » ... to each node, then the overall performance of Hadoop may be reduced. The majorreasonis thatdifferentcomputing capacitiesbetweennodes causethetask executiontimeto differ so thatthefasterexecutionrate nodes processinglocal data blocks faster than other slower nodes do.The required data should be transferredfrom another node through the network.Becausewaitingforthedatatransmissiontimeincreasesthetask executiontime,it causestheentirejobexecution timeto becomeprolonged.
doi:10.14419/ijet.v7i2.4.10034 fatcat:vovw2qjl7ng3jp2rzhgpyeuuxm