A Novel Priority Based Hadoop Energy Efficient Job Scheduling and Migration Technique with Multi Level Queue on YARN Scheduler

G. Joel Sunny Deol, Dr.NagaRaju O
2019 Helix  
The process or the framework for MapReduce works in two parts as Mapper and Reducer. The reducer algorithm analyzes the input from the tasks characteristics and generate recommendations for the applicable allocation of the work and the mapper algorithm analyses the perfect or the best fit for the task or the programming running on the Hadoop clusters. The primary challenge is to manage the migration of the virtual machines to make these arrangements suitable to the Hadoop scheduling
more » ... . Hence the demand from the research is to justly the Hadoop scheduling capabilities and test the performances of the scheduler strategies for diversified workloads. Also, it is important to design a virtual machine migration algorithm to justify the demands of low power consumptions. Accordingly, this work also coined an energy efficient technique for Hadoop MapReduce jobs scheduling and migration technique. The work results into a novel algorithm and provide significant improvement of the energy consumption. The outcome of the work also analyzes the improvement of other performance parameters like identification of ill-scheduled job and total execution time. This work demonstrates a significant 30% reduction of energy with nearly 40% reduction in job identification and migration time.
doi:10.29042/2019-4864-4869 fatcat:pqu7nzxwanahtiaq3s3fgz6bji