A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2006; you can also visit the original URL.
The file type is application/pdf.
Filters
Cluster scheduling for explicitly-speculative tasks
2004
Proceedings of the 18th annual international conference on Supercomputing - ICS '04
Preserved Fulltext
Large-scale computing often consists of many speculative tasks to test hypotheses, search for insights, and review potentially finished products. ...
which tasks are speculative. ...
explicitly supports this model. ...
doi:10.1145/1006209.1006256
dblp:conf/ics/PetrouGG04
fatcat:xodvih624rhdnnx2dxxllvsgtq
Improving MapReduce Performance in Heterogeneous Network Environments and Resource Utilization
2012
2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (ccgrid 2012)
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2012; you can also visit the original URL.
The file type is application/pdf.
We investigate network heterogeneity aware scheduling of both map and reduce tasks. ...
To alleviate the issue, we propose Benefit Aware Speculative Execution which predicts the benefit of launching new speculative tasks and greatly eliminates unnecessary runs of speculative tasks. ...
With default policy, Hadoop will launch a speculative task T 1 ' for T 1 . We assume T 1 ' and T 2 progress equally fast. ...
doi:10.1109/ccgrid.2012.12
dblp:conf/ccgrid/GuoF12
fatcat:7cmx4wtff5g2dcsjpzdjqiqfqy
Optimization for Speculative Execution of Multiple Jobs in a MapReduce-like Cluster
[article]
2015
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
In this paper, we focus on the design of speculative execution schemes for a parallel processing cluster under different loading conditions. ...
We also derive the workload threshold under which SCA or SDA should be used for speculative execution. ...
Input: The jobs in the cluster associated with their running status at time slot l; Output: Scheduling decisions for time slot l. 1 schedule the unassigned tasks of the running jobs in the cluster with ...
arXiv:1406.0609v3
fatcat:ept2yi2xabhs3ldn5r6heag2c4
Asynchronous Non-Blocking Algorithm to Handle Straggler Reduce Tasks in Hadoop System
2020
International Journal on Advanced Science, Engineering and Information Technology
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
When the task is ready to run, it will be sent back to the main task queue for running. ...
However, the algorithms manage Map and Reduce task similarly, while the straggler root cause might be different for both tasks. ...
The scheduler is a good default for small to medium-sized clusters. ...
doi:10.18517/ijaseit.10.5.9073
fatcat:ope67evg3rcdjnvevsqxuwxeiq
Performance-aware Speculative Resource Oversubscription for Large-scale Clusters
2020
IEEE Transactions on Parallel and Distributed Systems
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Instead of waiting for resource allocation to be confirmed by the centralized scheduler, job managers in ROSE can independently request to launch speculative tasks within specific machines according to ...
It is a long-standing challenge to achieve a high degree of resource utilization in cluster scheduling. ...
ACKNOWLEDGMENT We would like to thank Alibaba Group, particularly colleagues from Fuxi scheduling team for their collaboration. ...
doi:10.1109/tpds.2020.2970013
fatcat:2tftu6ehofcqflyfhumaytry7u
Task-Cloning Algorithms in a MapReduce Cluster with Competitive Performance Bounds
[article]
2015
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Job scheduling for a MapReduce cluster has been an active research topic in recent years. ...
cluster based on the Shortest Remaining Processing Time scheduler (SRPT). ...
A stochastic program formulation for job scheduling For any job, all the map tasks and reduce tasks can only be scheduled after the job arrival at the cluster and hence Finally, a job completes when all ...
arXiv:1501.02330v1
fatcat:n5hoiptz7benvp3jjqg32txnvq
Optimization strategies for A/B testing on HADOOP
2013
Proceedings of the VLDB Endowment
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL.
The file type is application/pdf.
Our proposed solution relies on proper resource allocation for concurrent Hive jobs based on data dependency, inter-query optimization and modeling of Hadoop cluster load. ...
This work was performed on eBay production Hadoop cluster. ...
We thank Tony Thrall, manager of the EDA team at eBay Inc. for permission to publish the results of this study. ...
doi:10.14778/2536222.2536224
fatcat:hu7hwxhcfbf73ka2wbyrnkt4wq
Effective Straggler Mitigation: Attack of the Clones
2013
Symposium on Networked Systems Design and Implementation
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Small jobs, that are typically run for interactive data analyses in datacenters, continue to be plagued by disproportionately long-running tasks called stragglers. ...
slower than the median task in that job. ...
When a job is submitted, its tasks are queued at the scheduler. For every queued task, the scheduler spawns many clones. ...
dblp:conf/nsdi/AnanthanarayananGSS13
fatcat:5hmptbtvvbanhdnr6ibjvv42ga
Wrangler
2014
Proceedings of the ACM Symposium on Cloud Computing - SOCC '14
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2015; you can also visit the original URL.
The file type is application/pdf.
In particular, by using these predictions to balance delay in task scheduling against the potential for idling of resources, Wrangler achieves a speed up in the overall job completion time. ...
For production-level workloads from Facebook and Cloudera's customers, Wrangler improves the 99 th percentile job completion time by up to 61% as compared to speculative execution, a widely used straggler ...
We also thank our shepherd, Fred Douglis, for help in shaping the final version of the paper. ...
doi:10.1145/2670979.2671005
dblp:conf/cloud/YadwadkarAK14
fatcat:rfwiymfinrdj3iq4446ogppvee
The Case for Tiny Tasks in Compute Clusters
2013
USENIX Workshop on Hot Topics in Operating Systems
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL.
The file type is application/pdf.
We argue for breaking data-parallel jobs in compute clusters into tiny tasks that each complete in hundreds of milliseconds. ...
Tiny tasks avoid the need for complex skew mitigation techniques: by breaking a large job into millions of tiny tasks, work will be evenly spread over available resources by the scheduler. ...
Acknowledgments We thank Matei Zaharia, Colin Scott, John Ousterhout, and Patrick Wendell for useful feedback on earlier drafts of this paper. ...
dblp:conf/hotos/OusterhoutPRVXRSS13
fatcat:7jn7fbnrgrclvhinapthwhs2oa
Privacy Preservation in Analyzing EHealth Records in Big Data Environment
2015
International Journal on Recent and Innovation Trends in Computing and Communication
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2018; you can also visit the original URL.
The file type is application/pdf.
Third, data locality can be improved without any impact on fairness using Slot Pre Scheduling. ...
First, the under-utilization of map and reduce tasks is improved based on Dynamic Hadoop Slot Allocation (DHSA). ...
Zaharia.M et al. achieved locality and fairness [18] in cluster scheduling using delay scheduling. ...
doi:10.17762/ijritcc2321-8169.1504139
fatcat:357gk2cdnneljcdihzyoa2cbkm
Intelligent Resource Scheduling at Scale: A Machine Learning Perspective
2018
2018 IEEE Symposium on Service-Oriented System Engineering (SOSE)
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2018; you can also visit the original URL.
The file type is application/pdf.
The exhibited heterogeneity of workload and server characteristics in Cloud-scale or Internetscale environments has raised unprecedented new challenges for cluster scheduling. ...
Compared with ad-hoc heuristics for a multi-resource cluster scheduling problem, machine learning (ML) approaches can in turn facilitate improved efficiency in resource management. ...
However, its performance can be undermined due to the fact that speculative copies could occupy available resources for new tasks. ...
doi:10.1109/sose.2018.00025
dblp:conf/sose/YangOCTX18
fatcat:yxyi5lwxmjdtdpqbrvblbqan5u
Cloud Resource Allocation as Preemptive Scheduling Approach
2014
International Journal of Computer Applications
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL.
The file type is application/pdf.
To achieve this many scheme are proposed, Nephele is one of the data processing framework which exploits the dynamic resource allocation offered by IaaS clouds for both task scheduling and execution. ...
This paper introduces a new Approach for increasing the efficiency of the scheduling algorithm for the real time Cloud Computing services. ...
Nephele is the data processing framework to explicitly exploit the dynamic resource allocation for both task scheduling and execution. ...
doi:10.5120/15452-3989
fatcat:dwzbdkziunbnhixsrzuzld247q
Fault Tolerance in MapReduce: A Survey
[chapter]
2016
Computer Communications and Networks
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
of these tasks on other machines. ...
Furthermore, in order to mask temporary failures caused by network or machine overload (timing failure) where some tasks are performing relatively slower than other tasks, Hadoop relaunches other copies ...
Greedy Speculative Scheduling (GS). This algorithm is intended to greedily pick a task that will be scheduled next. 2. Resource Aware Speculative Scheduling (RAS). ...
doi:10.1007/978-3-319-44881-7_11
dblp:series/ccn/MemishiIPA16
fatcat:m5x33gpzunhzzgrdslagndiwzy
Observations on Factors Affecting Performance of MapReduce based Apriori on Hadoop Cluster
[article]
2017
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
Other Versions
2016
2016 International Conference on Computing, Communication and Automation (ICCCA)
unpublished
doi:10.1109/ccaa.2016.7813695 fatcat:hvrfxr7xbzgfpgqvqeoz3caehy
doi:10.1109/ccaa.2016.7813695 fatcat:hvrfxr7xbzgfpgqvqeoz3caehy
Designing fast and scalable algorithm for mining frequent itemsets is always being a most eminent and promising problem of data mining. ...
The non-algorithmic factors include speculative execution, nodes with poor performance, data locality & distribution of data blocks, and parallelism control with input split size. ...
Fig. 3 it can be seen that speculative execution comes in action only for a job taking longer time to complete. Speculative task is not launched for short jobs. ...
arXiv:1701.05982v1
fatcat:mt3nog3purhwbkh2pxwnhoc75a
« Previous
Showing results 1 — 15 out of 14,941 results