InterFS

Peng Wang, Le Cao, Chunbo Lai, Leqi Zou, Guangyu Sun, Jason Cong
2015 Proceedings of the 6th Asia-Pacific Workshop on Systems - APSys '15  
Resource under-utilization is a common problem in modern data centers. Though researchers have proposed consolidation techniques to improve utilization of computing resources, there still lacks an approach to mitigate particularly low utilization of storage capacity in clusters for online services. A potential solution is to "interplant" a distributed storage system together with the services on these clusters to leverage the unused storage. However, avoiding performance interference with
more » ... ng services is an essential prerequisite for interplanting. Thus, we propose InterFS, a POSIX-compliant distributed file system aiming at fully exploiting the storage resource on data center clusters. We adopt intelligent resource isolation, peak load dodging, and region-based replica placement schemes in InterFS. Therefore, it can be interplanted with other resource-intensive services without interfering with them, and amply fulfill the storage requirements of small-scale applications in the data center. Currently InterFS is deployed in 20,000+ servers at Baidu, providing 80 PB storage space to 200+ long-tailed services.
doi:10.1145/2797022.2797036 dblp:conf/apsys/WangCLZSC15 fatcat:5m75ai2jpbaltantmo7hncr6jy