Omega

Malte Schwarzkopf, Andy Konwinski, Michael Abd-El-Malek, John Wilkes
2013 Proceedings of the 8th ACM European Conference on Computer Systems - EuroSys '13  
Increasing scale and the need for rapid response to changing requirements are hard to meet with current monolithic cluster scheduler architectures. This restricts the rate at which new features can be deployed, decreases efficiency and utilization, and will eventually limit cluster growth. We present a novel approach to address these needs using parallelism, shared state, and lock-free optimistic concurrency control. We compare this approach to existing cluster scheduler designs, evaluate how
more » ... ch interference between schedulers occurs and how much it matters in practice, present some techniques to alleviate it, and finally discuss a use case highlighting the advantages of our approach -all driven by real-life Google production workloads. Monolithic Two-level Shared state cluster machines cluster state information scheduling logic 1 In the public trace for cluster C, these are priority bands 0-8 [27] .
doi:10.1145/2465351.2465386 dblp:conf/eurosys/SchwarzkopfKAW13 fatcat:onvfyrf6ybbobgplgtg6eyndcm