Hadoop Periodic Jobs Using Data Blocks to Achieve Efficiency

Sujit Roy, Subrata Das, Indrani Mandal
unpublished
To manage, process, and analyze very large datasets, HADOOP has been a powerful, fault-tolerant platform. HADOOPis used to access big data because it is effective, scalable and is well supported by large trafficker and user communities. This research paper proposed a new approach to process the data in HADOOP to achieve the efficiency of data processing by using synchronous data transmission, sending block of data from source to destination. Here a method has been shown how to divide the data
more » ... ocks in achieving optimal efficacy by adjusting the split size or using appropriate size of staffs. As the effective HADOOP hardware configuration matches the requirements of each periodic task, so this allows our system to the data blocks increasing data efficiency as well as throughput. Finally, experiments showed the effectiveness of these methods with high data efficiency (around 22% more than existing system), low installation cost and the feasibility of this method.
fatcat:iezqll6x3bhi5a7gxdxt26x2wq