A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2016; you can also visit the original URL.
The file type is application/pdf.
Filters
Evaluating task scheduling in hadoop-based cloud systems
2013
2013 IEEE International Conference on Big Data
Preserved Fulltext
In this paper, we use a recently-released Cloud benchmarks suite---CloudRank-D to quantitatively evaluate five different Hadoop task schedulers, including FIFO, capacity, naïve fair sharing, fair sharing ...
Nowadays, private clouds are widely used for resource sharing. Hadoop-based clusters are the most popular implementations for private clouds. ...
In the future, we will use more complex workloads to study and evaluate more efficient task schedulers for Hadoop based cloud systems. ...
doi:10.1109/bigdata.2013.6691697
dblp:conf/bigdataconf/LiuXLL13
fatcat:tpvvmfinmnfp3mimfldi6bsdw4
Dynamic Load Aware Scheduler of Map Reduce Tasks for Cloud Environments
2019
International journal of recent technology and engineering
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
The execution times of Hadoop on cloud can be improved if the virtual resources are effectively used to schedule the tasks by studying the resource usage characteristics of the tasks and resource availability ...
The proposed work is to build a dynamic scheduler for Hadoop framework which can make scheduling decision dynamically based on job resource usage and node load. ...
EVALUATION AND RESULTS The proposed dynamic load aware scheduler for Hadoop is evaluated on private and public cloud environments. ...
doi:10.35940/ijrte.b1079.0982s1119
fatcat:rrjdnrgikfdujigtnitzpeona4
Locality-aware dynamic VM reconfiguration on MapReduce clouds
2012
Proceedings of the 21st international symposium on High-Performance Parallel and Distributed Computing - HPDC '12
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL.
The file type is application/pdf.
Cloud computing based on system virtualization, has been expanding its services to distributed data-intensive platforms such as MapReduce and Hadoop. ...
DRR dynamically increases or decreases the computing capability of each node to enhance locality-aware task scheduling. ...
There have been several recent studies to manage virtual resources efficiently in cloud systems [8] . Distributed Resource Scheduler (DRS) proposed a cloud-scale resource management system [6] . ...
doi:10.1145/2287076.2287082
dblp:conf/hpdc/ParkLKHM12
fatcat:hvb5mh7hprbehj2n5dqdg4n2u4
SAGS: A SLA-Aware Green Scheduling in Heterogeneous Cloud Using Hadoop YARN
2018
International Journal of Intelligent Engineering and Systems
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Hence, this paper proposes the problem of energy-aware heterogeneous Hadoop Yarn cloud with deadline based SLA. ...
We proposed a SLA-Aware Green Scheduling (SAGS), a Dynamic Voltage/Frequency Scaling (DVFS) based approach along with SLA-Aware scheduling algorithm in the heterogeneous environment. ...
Our contributions are summarized below,
Energy-Aware Eco-friendly SLA-Based scheduler for the heterogeneous environment in the cloud. 2. SAGS is implemented in the Hadoop 2. x open source. 3. ...
doi:10.22266/ijies2018.1231.11
fatcat:2nqd6fdrjjeadfdtzqxejfayui
An Accurate and Efficient Scheduler for Hadoop MapReduce Framework
2018
Indonesian Journal of Electrical Engineering and Computer Science
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
This work present a novel makespan model for Hadoop MapReduce framework namely OHMR (Optimized Hadoop MapReduce) to process data in real-time and utilize system resource efficiently. ...
However, the critical challenges of cloud service provider is to meet user task Service level agreement (SLA) requirement (task deadline). ...
In [13] , an AffordHadoop application is adopted to reduce cost in finishing various tasks and to allocate data and schedule tasks and hence efficiency of system get enhanced. ...
doi:10.11591/ijeecs.v12.i3.pp1132-1142
fatcat:oi4s5wxtr5hwbbw4heh6zhuqda
An empirical analysis of scheduling techniques for real-time cloud-based data processing
2011
2011 IEEE International Conference on Service-Oriented Computing and Applications (SOCA)
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2012; you can also visit the original URL.
The file type is application/pdf.
Our evaluations on Amazon EC2 show that the existing Hadoop scheduler is ill-equipped to handle jobs with deadlines. ...
To highlight the challenges in this space, we provide a case study of the online scheduling of MapReduce jobs executed by Hadoop. ...
Existing Scheduling Policies in Hadoop In the current Hadoop implementation, scheduling of MapReduce jobs proceeds as follows. ...
doi:10.1109/soca.2011.6166240
dblp:conf/soca/PhanZZLL11
fatcat:33qzngu6dbfyvg2qhzit7g5hiy
Optimized memory model for hadoop map reduce framework
2019
International Journal of Electrical and Computer Engineering (IJECE)
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
This work present a novel memory optimization model for Hadoop Map Reduce framework namely MOHMR (Optimized Hadoop Map Reduce) to process data in real-time and utilize system resource efficiently. ...
However, the critical challenges of cloud service provider is to meet user task Service level agreement (SLA) requirement (task deadline). ...
In [13] , an Afford-Hadoop application is adopted to reduce cost in finishing various tasks and to allocate data and schedule tasks and hence efficiency of system get enhanced. ...
doi:10.11591/ijece.v9i5.pp4396-4407
fatcat:hrlfba645feivf7uss7jjpy7xu
A Hybrid Scheduling Approach for Scalable Heterogeneous Hadoop Systems
2012
2012 SC Companion: High Performance Computing, Networking Storage and Analysis
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2015; you can also visit the original URL.
The file type is application/pdf.
There is a considerable challenge to schedule the growing number of tasks and resources in a scalable manner. ...
This solution is being used widely by most Cloud providers. Hadoop schedulers are critical elements for providing desired performance levels. A scheduler assigns MapReduce tasks to Hadoop resources. ...
for permission to use their workload traces in this research. ...
doi:10.1109/sc.companion.2012.155
dblp:conf/sc/OskooeiD12
fatcat:lsg2mdbmkjgxpojr24xbpj74z4
Failure Analysis of Hadoop Schedulers using an Integration of Model Checking and Simulation
2021
Electronic Proceedings in Theoretical Computer Science
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Note that this fulltext copy is not of the "primary" version of this work. The version it corresponds to is:
2021 pre-print arXiv:2109.04196v1 fatcat:lsaub6cg2bhhhimc4rbivobokq
The Hadoop scheduler is a centerpiece of Hadoop, the leading processing framework for data-intensive applications in the cloud. ...
Next, we use the proposed formal model to analyze the scheduler of OpenCloud, a Hadoop-based cluster that simulates the Hadoop load, in order to illustrate the usability and benefits of our work. ...
In the cloud, failures are the norm rather than the exception and they often impact the performance of tasks running in Hadoop. ...
doi:10.4204/eptcs.342.10
fatcat:nqsbz2rpxjdndo56ppxpmjlamy
Accelerating MapReduce Analytics Using CometCloud
2012
2012 IEEE Fifth International Conference on Cloud Computing
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL.
The file type is application/pdf.
Furthermore, we develop an autonomic manager that can schedule MapReduce tasks based on user objective, provision resources accordingly, and support on-demand scale up and cloudbursts. ...
Heterogeneity is also unavoidable in scientific applications that process a varying number of datasets of different sizes. In these cases, the performance of MapReduce-Hadoop can be a concern. ...
The research presented in this paper is supported in part by National Science Foundation via grants numbers IIP 0758566, CCF-0833039, DMS-0835436, CNS 0426354, IIS 0430826, and CNS 0723594, by Department ...
doi:10.1109/cloud.2012.150
dblp:conf/IEEEcloud/AbdelBakyKRP12
fatcat:oxqjtykasbffxlhgjgniojb6me
CloudRank-D: benchmarking and ranking cloud computing systems for data processing applications
2012
Frontiers of Computer Science
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL.
The file type is application/pdf.
In several case studies, we evaluate two small-scale deployments of cloud computing systems using CloudRank-D. ...
However, there is no existing benchmark suite for evaluating cloud performance on the whole system level. ...
Hadoop schedulers The scheduler in Hadoop is a pluggable component, and system managers can customize the scheduler according their requirements. There are three schedulers in Hadoop 0.20.2. ...
doi:10.1007/s11704-012-2118-7
fatcat:jatll7tuuvf33dfxssntrowmau
Evaluating Virtualization for Hadoop MapReduce on an OpenNebula Cloud
2014
International Journal of Multimedia and Image Processing
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2018; you can also visit the original URL.
The file type is application/pdf.
In this work, we conducted benchmarks to measure the performance of a Hadoop cluster deployed on OpenNebula clouds with KVM and OpenVZ. ...
In this context, virtualization has been used as a platform for resource-intensive applications, like Hadoop, as it has brought features like server consolidation, scalability and better resources usage ...
Experimental Evaluation To evaluate the performance of Hadoop cluster on each virtualization platform, we propose the establishment of two private OpenNebula clouds. ...
doi:10.20533/ijmip.2042.4647.2014.0029
fatcat:upuyub3fabcq7gtvpsqqyu57sq
Performance evaluation and resource optimization of cloud based parallel Hadoop clusters with an intelligent scheduler
2018
International Journal of Engineering & Technology
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Processing of such a huge incremental data in large scale requires a parallel processing system like Hadoop based cluster. ...
Major challenge that arises in all cluster-based system is how efficiently the resources of the system can be used. ...
As cloud took over the storage profile for most of the information systems, private cloud came as solution for the workload balancing for the Hadoop based systems. ...
doi:10.14419/ijet.v7i4.13372
fatcat:sjoercq2mzaz7i3kiicourmtqy
LSTPD: Least Slack Time-Based Preemptive Deadline Constraint Scheduler for Hadoop Clusters
2020
IEEE Access
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
In order for better allocation of tasks and load balancing, we first analyze the task scheduling behaviors of the Hadoop platform. ...
In this paper, we propose an efficient preemptive deadline constraint scheduler based on least slack time and data locality. ...
PERFORMANCES EVALUATION In this section, we describe our experimental setup for evaluating the proposed LSTPD scheduling scheme by using Hadoop 2.7.2 (stable), Linux operating system, and Cen-tOS. ...
doi:10.1109/access.2020.3002565
fatcat:yvvxyliqavakfkmti5sim7tv3a
Empirical Study of Job Scheduling Algorithms in Hadoop MapReduce
2017
Cybernetics and Information Technologies
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
This article focuses on empirically evaluating the performance of three schedulers: First In First Out (FIFO), Fair scheduler, and Capacity scheduler. ...
To carry out the experimental evaluation, we implement our own Hadoop cluster testbed, consisting of four machines, in which one of the machines works as the master node and all four machines work as slave ...
In [8] , Cloud benchmark suite CloudRank-D is used to evaluate different Hadoop job schedulers, which include FIFO scheduler, Fair scheduler, Fair scheduler with delay, Capacity scheduler, and Hadoop ...
doi:10.1515/cait-2017-0012
fatcat:kocust3jjfcqjl4thmkbcouiii
« Previous
Showing results 1 — 15 out of 5,368 results