Scalable and Sustainable Long Term Digital Preservation of Scientific Datasets

Matthew Addis
2021 International Conference on Digital Preservation  
The European Commission supported ARCHIVER project (Archiving and Preservation for Research Environments) aims to "introduce significant improvements in the area of archiving and digital preservation services, supporting the IT requirements of European scientists and providing end-to-end archival and preservation services, cost-effective for data generated in the petabyte range with high, sustained ingest rates, in the context of scientific research projects". This paper presents a software
more » ... tion developed by Arkivum to meet the needs of long-term digital preservation of scientific datasets in ARCHIVER. We present and discuss how this solution is scalable (able to process and store very large volumes of research data) and sustainable (both economically and environmentally). This is achieved through a combination of serverless computing, deployment on hyperscale infrastructure, and implementation of configurable 'Minimum Effort Ingest' workflows. In particular, we show how high-performance and scalable Long Term Digital Preservation (LTDP) of verylarge datasets can be done in a way that is entirely compatible with high levels of cost-efficiency and minimized environmental impact.
dblp:conf/ipres/Addis21a fatcat:dbxpx7i26nfvdefc4vsyx67c5q