Petabyte-scale data migration at CERNBox

Hugo Gonzalez Labrador, Jose Ramon Mendez Reboredo
2019 Zenodo  
The FDO section operates and supports the storage and file system ser- vices for physics. I joined the FDO section as Technical Student in 2014 and currently I am a Staff member of the section. My main role is being the Service Manager for CERNBox, the CERN biggest cloud storage system. My main activity is to pro-actively manage the service and the team of twelve people that provides a Dropbox-like solution to more than 17,000 users and 200 institutions with a yearly growth of 400 percent.
more » ... activities involve day-to-day storage operations and end-user support for CERN back- bone storage technologies: EOS, CASTOR and CERNBox, managing more than 1,500 servers with more than 60,000 disks, representing about 300 petabytes of data. This thesis focuses on the design and analysis of the current living system to support a major data migration that will happen in the next months. I decided to take the opportunity to use this activity as the main source for this thesis. The focus of the thesis is the architecture and re-architecture of the systems involved in the CERNBox Service to accommodate the migration. The thesis will not focus on a particular software implementation but rather on high-level design and technological choices to ensure the migration is a success.
doi:10.5281/zenodo.3402900 fatcat:i2ywssmct5fpjej4e2uuik5s2e