Reproducible Containers

Omar S. Navarro Leija, Kelly Shiptoski, Ryan G. Scott, Baojun Wang, Nicholas Renner, Ryan R. Newton, Joseph Devietti
2020 Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems  
We describe the design and implementation of DetTrace, a reproducible container abstraction for Linux implemented in user space. All computation that occurs inside a DetTrace container is a pure function of the initial filesystem state of the container. Reproducible containers can be used for a variety of purposes, including replication for fault-tolerance, reproducible software builds and reproducible data analytics. We use DetTrace to achieve, in an automatic fashion, reproducibility for
more » ... 0 Debian package builds, containing over 800 million lines of code, as well as bioinformatics and machine learning workflows. We show that, while software in each of these domains is initially irreproducible, DetTrace brings reproducibility without requiring any hardware, OS or application changes. DetTrace's performance is dictated by the frequency of system calls: IO-intensive software builds have an average overhead of 3.49×, while a compute-bound bioinformatics workflow is under 2%. CCS Concepts. Software and its engineering Multiprocessing / multiprogramming / multitasking.
doi:10.1145/3373376.3378519 dblp:conf/asplos/LeijaSSWRND20 fatcat:234ucu6hrzbajahchp5y2tud7a