367 Hits in 4.7 sec

Energy-Aware Task Scheduling Using Hybrid Firefly-BAT (FFABAT) in Big Data

M. Senthilkumar
2018 Cybernetics and Information Technologies  
The aim of this study is to get the task-scheduling over Big data using Hadoop. Initially, we prioritize the tasks with the help of k-means clustering algorithm.  ...  This bat algorithm is established on the echo-location behaviour of micro-bats with variable pulse rates of emission and loudness.  ...  The HDFS results are compared to FA with the hybrid algorithm (FFA with BAT). Energy optimization is achieved by using the hybrid algorithm FFA-BAT. e r, I., X. S. Y a n g, J.  ... 
doi:10.2478/cait-2018-0031 fatcat:ladjjjfwlfhgrjx7dtmyh56yha

Survey on improved Autoscaling in Hadoop into cloud environments

Masoumeh Rezaei Jam, Leyli Mohammad Khanli, Mohammad Kazem Akbari, Elham Hormozi, Morteza Sargolzaei Javan
2013 The 5th Conference on Information and Knowledge Technology  
Based on the evaluation methods we understand that "The controller module and BEEMR" are best way to improve energy performance.  ...  Running Hadoop on a cloud means that we have the facilities to add or remove computing power from the Hadoop cluster within minutes or even less by provisioning more machines or shutting down currently  ...  Fig. 5 shows a 24% reduction in energy consumption of a 1560 server datacenter with 80% capacity utilization [16] .  ... 
doi:10.1109/ikt.2013.6620031 fatcat:3ra3zakwgrd2zaeelc3h3jtrqe

Big vs little core for energy-efficient Hadoop computing

Maria Malik, Katayoun Neshatpour, Tinoosh Mohsenin, Avesta Sasan, Houman Homayoun
2017 Design, Automation & Test in Europe Conference & Exhibition (DATE), 2017  
Heterogeneous architectures that combine big Xeon cores with little Atom cores have emerged as a promising solution to enhance energy-efficiency by allowing each application to run on an architecture that  ...  In this paper, we characterize Hadoop-based applications and their corresponding MapReduce tasks on big Xeon and little Atom-based server architectures to understand how the choice of big vs little cores  ...  Considering the power consumption of Xeon, Atom cores will be a more efficient choice to execute the Hadoop applications.  ... 
doi:10.23919/date.2017.7927225 dblp:conf/date/MalikNMSH17 fatcat:vpysnphlcvehfgjtre7g7wakli

Analysis and evaluation of MapReduce solutions on an HPC cluster

Jorge Veiga, Roberto R. Expósito, Guillermo L. Taboada, Juan Touriño
2016 Computers & electrical engineering  
This work aims to establish a taxonomy of these frameworks together with a thorough evaluation, which has been carried out in terms of performance and energy efficiency metrics.  ...  Moreover, Big Data frameworks like Hadoop need to be adapted to leverage the available resources in HPC environments.  ...  energy consumption of RDMA-Hadoop-2 (see Figure 5d ).  ... 
doi:10.1016/j.compeleceng.2015.11.021 fatcat:d3n3fkxit5hypdb5mlwh2j7wiy

Evaluation and Analysis of GreenHDFS: A Self-Adaptive, Energy-Conserving Variant of the Hadoop Distributed File System

Rini T. Kaushik, Milind Bhandarkar, Klara Nahrstedt
2010 2010 IEEE Second International Conference on Cloud Computing Technology and Science  
We present a detailed evaluation and sensitivity analysis of an energy-conserving, highly scalable variant of the Hadoop Distributed File System (HDFS) called Green-HDFS.  ...  HDFS traces show that GreenHDFS can achieve 24% energy cost reduction by doing power management in only one top-level tenant directory in the cluster and meets all the scale-down mandates in spite of the  ...  Figure 9 (Left) shows a 24% reduction in energy consumption of a 1560 server datacenter with 80% capacity utilization.  ... 
doi:10.1109/cloudcom.2010.109 dblp:conf/cloudcom/KaushikBN10 fatcat:5e3s6yauh5hgdatyor6qu4f7em

A Novel Priority Based Hadoop Energy Efficient Job Scheduling and Migration Technique with Multi Level Queue on YARN Scheduler

G. Joel Sunny Deol, Dr.NagaRaju O
2019 Helix  
This work demonstrates a significant 30% reduction of energy with nearly 40% reduction in job identification and migration time.  ...  The work results into a novel algorithm and provide significant improvement of the energy consumption.  ...  with comprehension of the importance in time and information multifaceted nature worldview like FIFO scheduler, Fair Scheduler, Capacity Scheduler, Dynamic Priority Based Hybrid Scheduler, Name Performance  ... 
doi:10.29042/2019-4864-4869 fatcat:pqu7nzxwanahtiaq3s3fgz6bji

A comprehensive view of Hadoop research—A systematic literature review

Ivanilton Polato, Reginaldo Ré, Alfredo Goldman, Fabio Kon
2014 Journal of Network and Computer Applications  
Lately, Apache Hadoop has attracted strong attention due to its applicability to Big Data processing.  ...  Solution: We conducted a systematic literature review to assess research contributions to Apache Hadoop.  ...  Table A1 and A2 Table A1 Studies with implementation and/or experiments (MapReduce and data storage & manipulation categories). Appendix A.  ... 
doi:10.1016/j.jnca.2014.07.022 fatcat:4xjveqy6mrctzjc4ou7llyy4u4

A Survey on Job Scheduling in Big Data

M. Senthilkumar, P. Ilango
2016 Cybernetics and Information Technologies  
Various scheduling algorithms of the MapReduce model using Hadoop vary with design and behavior, and are used for handling many issues like data locality, awareness with resource, energy and time.  ...  Big Data Applications with Scheduling becomes an active research area in last three years. The Hadoop framework becomes very popular and most used frameworks in a distributed data processing.  ...  Overall cost of energy increases in the data center, so minimizing the energy consumption becomes a big challenge in data centers.  ... 
doi:10.1515/cait-2016-0033 fatcat:psrtc3l3dzgkfklqjl6hr3qczi

Literature Survey of Dynamic Data Replication in Cloud Computing

K. Sasikumar, Madiajagan Muthaiyan
2016 Research Journal of Applied Sciences Engineering and Technology  
All replication technique access some attributes such as fault tolerance, scalability, reliability, performance, storage consumption, data access time etc.  ...  Techniques based on Hadoop Distributed File System (HDFS): A number of earlier works related to data replication that employs in Hadoop Distributed File System (HDFS) architecture in the literature inspired  ...  The Hadoop Distributed File System (HDFS) was one such technology that administers the Big Data competently.  ... 
doi:10.19026/rjaset.13.2927 fatcat:dfxivck7bjg7zhdp2bjhremvti

Multi-Layer-Mesh: A Novel Topology and SDN-Based Path Switching for Big Data Cluster Networks

Leandro Batista de Almeida, Damien Magoni, Philip Perry, Eduardo Cunha de Almeida, John Murphy, Anthony Ventresque
2019 ICC 2019 - 2019 IEEE International Conference on Communications (ICC)  
Acknowledgement This work was supported with the financial support of the Science Foundation Ireland grant 13/RC/2094 to Lero -the Irish Software Research Centre (  ...  Even more significant is the potential cut in costs, related with the energy consumption from the switches and the related cooling systems.  ...  Thus, it reduces costs both in equipment acquisition and maintenance, and in energy consumption and cooling.  ... 
doi:10.1109/icc.2019.8761785 dblp:conf/icc/AlmeidaMPA0V19 fatcat:gqjulpw67replnco5j34gbn3qe

A Survey of Big Data Machine Learning Applications Optimization in Cloud Data Centers and Networks [article]

Sanaa Hamid Mohamed, Taisir E.H. El-Gorashi, Jaafar M.H. Elmirghani
2019 arXiv   pre-print
Moreover, enterprises with multiple tenants requesting various big data services are challenged by the need to optimize leasing their resources at reduced running costs and power consumption while avoiding  ...  MapReduce and Hadoop thus introduce innovative, efficient, and accelerated intensive computations and analytics.  ...  Compared to Hadoop with YARN, a reduction by 35% in brown energy usage and by 21% in overall energy consumption was obtained. D.  ... 
arXiv:1910.00731v1 fatcat:kvi3br4iwzg3bi7fifpgyly7m4

HybridMR: A Hierarchical MapReduce Scheduler for Hybrid Data Centers

Bikash Sharma, Timothy Wood, Chita R. Das
2013 2013 IEEE 33rd International Conference on Distributed Computing Systems  
By consolidating batch jobs with over-provisioned foreground applications, the available unused resources are better utilized, resulting in improved application performance and energy efficiency.  ...  These results indicate that a hybrid data center with an efficient scheduling mechanism can provide a cost-effective solution for hosting both batch and interactive workloads.  ...  [12] proposed techniques for MapReduce provisioning in virtualized cloud clusters, with emphasis on energy reduction.  ... 
doi:10.1109/icdcs.2013.31 dblp:conf/icdcs/SharmaWD13 fatcat:aifrfiqwsbd73ldyzcx54zrreq

Hybrid Map Task Scheduling for GPU-Based Heterogeneous Clusters

Koichi Shirahata, Hitoshi Sato, Satoshi Matsuoka
2010 2010 IEEE Second International Conference on Cloud Computing Technology and Science  
To address this problem, we have proposed a hybrid scheduling technique for GPU-based computer clusters, which minimizes the execution time of a submitted job using dynamic profiles of Map tasks running  ...  We have implemented a prototype of our proposed scheduling technique by extending MapReduce framework, Hadoop.  ...  consumption.  ... 
doi:10.1109/cloudcom.2010.55 dblp:conf/cloudcom/ShirahataSM10 fatcat:46xq6lkpkbelfmhkggbddja7zu

Application of Big Data Fusion Based on Cloud Storage in Green Transportation: An Application of Healthcare

Li Qin Hu, Amit Yadav, Asif Khan, Hong Liu, Amin Ul Haq
2020 Scientific Programming  
However, with the continuous growth in the number of energy vehicles, it is necessary for the government to provide strong support in the construction of charging piles.  ...  Movements of the world towards green environments, there is hike in use and production of electric vehicles (energy vehicles).  ...  Compared with the United States and Japan, Europe focuses more on greenhouse gas reduction strategies.  ... 
doi:10.1155/2020/1593946 fatcat:lofhbyiorbhu3bmw5kjavyan6q

Analysis of power consumption in heterogeneous virtual machine environments

Catalin Negru, Mariana Mocanu, Valentin Cristea, Stelios Sotiriadis, Nik Bessis
2016 Soft Computing - A Fusion of Foundations, Methodologies and Applications  
Reduction of energy consumption in Cloud computing datacenters today is a hot a research topic, as these consume large amounts of energy.  ...  Based on the relation between server load and energy consumption, we study the eciency of data-intensive applications, and the penalties, in terms of power consumption, that are introduced by dierent degrees  ...  Acknowledgements The work has been funded by the projects: DataWay: This article does not contain any studies with human participants or animals performed by any of the authors.  ... 
doi:10.1007/s00500-016-2129-7 fatcat:ohul3jzatrfi3ixzocu5ofcp3a
« Previous Showing results 1 — 15 out of 367 results