Filters








152 Hits in 3.3 sec

Rucio: Scientific Data Management

Martin Barisits, Thomas Beermann, Frank Berghaus, Brian Bockelman, Joaquin Bogado, David Cameron, Dimitrios Christidis, Diego Ciangottini, Gancho Dimitrov, Markus Elsing, Vincent Garonne, Alessandro di Girolamo (+18 others)
<span title="2019-08-09">2019</span> <i title="Springer Science and Business Media LLC"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/jsovv4uypvfjtkbi66krm7jscq" style="color: black;">Computing and Software for Big Science</a> </i> &nbsp;
Rucio is an open-source software framework that provides scientific collaborations with the functionality to organize, manage, and access their data at scale.  ...  data organization, management, and access for scientific experiments which incorporates the existing tools and makes it easy to interact with them.  ...  We also thank former colleagues Miguel Branco, Pedro Salgado, and Florbela Viegas for their contributions to the Rucio predecessor system DQ2.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1007/s41781-019-0026-3">doi:10.1007/s41781-019-0026-3</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/3erfeeamhvg5ndqiiiibaunphm">fatcat:3erfeeamhvg5ndqiiiibaunphm</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20200311112843/https://link.springer.com/content/pdf/10.1007%2Fs41781-019-0026-3.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/1f/63/1f6378c51c7bc986d082eee2af2cd63f9ebd7c7a.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1007/s41781-019-0026-3"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> springer.com </button> </a>

Rucio - Scientific Data Management [article]

Martin Barisits, Thomas Beermann, Frank Berghaus, Brian Bockelman, Joaquin Bogado, David Cameron, Dimitrios Christidis, Diego Ciangottini, Gancho Dimitrov, Markus Elsing, Vincent Garonne, Alessandro di Girolamo, Luc Goossens, Wen Guan (+15 others)
<span title="2019-02-26">2019</span> <i > arXiv </i> &nbsp; <span class="release-stage" >pre-print</span>
Rucio is an open source software framework that provides scientific collaborations with the functionality to organize, manage, and access their volumes of data.  ...  Rucio has been originally developed to meet the requirements of the high-energy physics experiment ATLAS, and is continuously extended to support the LHC experiments and other diverse scientific communities  ...  We also thank former colleagues Miguel Branco, Pedro Salgado, and Florbela Viegas for their contributions to the Rucio predecessor system DQ2.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/1902.09857v1">arXiv:1902.09857v1</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/6rrkmqtlqrfq5koydykwv7vlnq">fatcat:6rrkmqtlqrfq5koydykwv7vlnq</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20200827042004/https://arxiv.org/pdf/1902.09857v1.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/74/37/74371ff977fbbc1da81486b572054a2b63217889.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/1902.09857v1" title="arxiv.org access"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> arxiv.org </button> </a>

Astronomical data organization, management and access in Scientific Data Lakes

Y.G. Grange, V.N. Pandey, X. Espinal, R. Di Maria, A.P. Millar
<span title="2022-02-03">2022</span> <i title="Zenodo"> Zenodo </i> &nbsp;
The scientific domain of particle physics also uses complex and distributed data management systems.  ...  Such a distributed data archive requires overarching data management orchestration.  ...  Scientific Data Lake The Scientific Data Lake infrastructure consists of several services that work together to manage the large data volumes from the LHC experiments as briefly illustrated in Fig. 1  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.5281/zenodo.6077555">doi:10.5281/zenodo.6077555</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/fpzc7uycknfhbmwndnbwg4u26a">fatcat:fpzc7uycknfhbmwndnbwg4u26a</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20220428033247/https://zenodo.org/record/6077556/files/X9-002.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/23/fc/23fcef6408629b1a6397cb76c96f5dd6e9538cef.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.5281/zenodo.6077555"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="unlock alternate icon" style="background-color: #fb971f;"></i> zenodo.org </button> </a>

Astronomical data organization, management and access in Scientific Data Lakes [article]

Y.G. Grange, V.N. Pandey, X. Espinal, R. Di Maria, A.P. Millar
<span title="2022-02-03">2022</span> <i > arXiv </i> &nbsp; <span class="release-stage" >pre-print</span>
The scientific domain of particle physics also uses complex and distributed data management systems.  ...  Such a distributed data archive requires overarching data management orchestration.  ...  Scientific Data Lake The Scientific Data Lake infrastructure consists of several services that work together to manage the large data volumes from the LHC experiments as briefly illustrated in Fig. 1  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2202.01828v1">arXiv:2202.01828v1</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/bxthhijjtrgelfbldhyhvc2y24">fatcat:bxthhijjtrgelfbldhyhvc2y24</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20220208200329/https://arxiv.org/pdf/2202.01828v1.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/21/d0/21d0554dc5dac2d61a806d1a66498c6dd2494b5d.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2202.01828v1" title="arxiv.org access"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> arxiv.org </button> </a>

ESCAPE Data Lake

Riccardo Di Maria, Rizart Dona, on behalf of the ESCAPE project, C. Biscarat, S. Campana, B. Hegner, S. Roiser, C.I. Rovelli, G.A. Stewart
<span title="">2021</span> <i title="EDP Sciences"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/oduy4e4fvfhovftixyndrlkomi" style="color: black;">EPJ Web of Conferences</a> </i> &nbsp;
The Pilot Data Lake consists of several storage services operated by the partner institutes and connected through reliable networks, and it adopts Rucio to orchestrate data management and organisation.  ...  This contribution describes the shared-ecosystem architecture of services, the Data Lake, fulfilling the needs in terms of data organisation, management, and access of the ESCAPE community.  ...  The ESCAPE Data Lake has been proven a successful scientific data management infrastructure where flagship ESFRIs in Astro-particle Physics, Electromagnetic and Gravitational-Wave Astronomy, Particle Physics  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1051/epjconf/202125102056">doi:10.1051/epjconf/202125102056</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/lt7kxbxyjnbujmmp6ytt2b3luu">fatcat:lt7kxbxyjnbujmmp6ytt2b3luu</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20210829222226/https://www.epj-conferences.org/articles/epjconf/pdf/2021/05/epjconf_chep2021_02056.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/61/ae/61ae8a99697ada3026b0438e47dac53cdd14fb59.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1051/epjconf/202125102056"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="unlock alternate icon" style="background-color: #fb971f;"></i> Publisher / doi.org </button> </a>

Transitioning CMS to Rucio Data Managment

Eric Vaandering, C. Doglioni, D. Kim, G.A. Stewart, L. Silvestris, P. Jackson, W. Kamleh
<span title="">2020</span> <i title="EDP Sciences"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/oduy4e4fvfhovftixyndrlkomi" style="color: black;">EPJ Web of Conferences</a> </i> &nbsp;
A description of CMS user data management with Rucio will also be provided.  ...  of data to Rucio.  ...  Rucio has met these challenges and will be deployed as the next CMS data management solution in 2020.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1051/epjconf/202024504033">doi:10.1051/epjconf/202024504033</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/b64hfhk7yvby5hzs6rjp7oiomi">fatcat:b64hfhk7yvby5hzs6rjp7oiomi</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20210428031049/https://lss.fnal.gov/archive/2020/conf/fermilab-conf-20-639-scd.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/13/54/13540af3e1f6518cb2c6b325bf64845ae82003d5.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1051/epjconf/202024504033"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="unlock alternate icon" style="background-color: #fb971f;"></i> Publisher / doi.org </button> </a>

Experience with Rucio in the wider HEP community

Martin Barisits, Thomas Beermann, David Cameron, James Alexander Clark, Riccardo Di Maria, Gabriele Gaetano Fronzé, Ian Johnson, Mario Lassnig, Cédric Serfon, Eric W Vaandering, C. Biscarat, S. Campana (+4 others)
<span title="">2021</span> <i title="EDP Sciences"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/oduy4e4fvfhovftixyndrlkomi" style="color: black;">EPJ Web of Conferences</a> </i> &nbsp;
In recent years, the scientific data management system Rucio has evolved into a successful open-source project that is now being used by many scientific communities and organisations.  ...  Managing the data of scientific projects is an increasingly complicated challenge, which was historically met by developing experiment-specific solutions.  ...  The open-source scientific data management system Rucio [1] has been developed to meet those objectives.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1051/epjconf/202125102030">doi:10.1051/epjconf/202125102030</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/petw3vxgffgirbj4qzbbmhk75a">fatcat:petw3vxgffgirbj4qzbbmhk75a</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20210825200818/https://www.epj-conferences.org/articles/epjconf/pdf/2021/05/epjconf_chep2021_02030.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/61/f5/61f527cdcc1b52fbc13de800595de440cc1f8c50.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1051/epjconf/202125102030"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="unlock alternate icon" style="background-color: #fb971f;"></i> Publisher / doi.org </button> </a>

Evaluating Rucio outside ATLAS: Common experiences from Belle II, CMS, DUNE, SKA, and LIGO

Eric Vaandering, James Clark, Mario Lassnig, Martin Barisits, Paul James Laycock, Robert Illingworth, Rohini Joshi
<span title="2019-11-04">2019</span> <i title="Zenodo"> Zenodo </i> &nbsp;
For many scientific projects, data management is an increasingly complicated challenge.  ...  To fulfill these needs, the data management system Rucio has been developed to allow the high-energy physics experiment ATLAS to manage its large volumes of data in an efficient and scalable way.  ...  is developing into a common standard for scientific data management ○ A successful collaborative open source projectRucio concepts -Namespace• All data stored in Rucio is identified by a Data IDentifier  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.5281/zenodo.3599074">doi:10.5281/zenodo.3599074</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/kbhs652wbrgp5adzfmub62gxlu">fatcat:kbhs652wbrgp5adzfmub62gxlu</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20200212021421/https://zenodo.org/record/3599075/files/CHEP2019_220.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/db/58/db5806a6dd103d691ea3a3480424511bb8c86ca0.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.5281/zenodo.3599074"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="unlock alternate icon" style="background-color: #fb971f;"></i> zenodo.org </button> </a>

Rucio beyond ATLAS: experiences from Belle II, CMS, DUNE, EISCAT3D, LIGO/VIRGO, SKA, XENON

Mario Lassnig, Martin Barisits, Paul J Laycock, Cédric Serfon, Eric W Vaandering, Katy Ellis, Robert A. Illingworth, Vincent Garonne, John White, James A. Clark, Gabriele Fronze, Rohini Joshi (+8 others)
<span title="">2020</span> <i title="EDP Sciences"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/oduy4e4fvfhovftixyndrlkomi" style="color: black;">EPJ Web of Conferences</a> </i> &nbsp;
For many scientific projects, data management is an increasingly complicated challenge.  ...  To fulfil these needs, the data management system Rucio has been developed to allow the high-energy physics experiment ATLAS at LHC to manage its large volumes of data in an efficient and scalable way.  ...  In conclusion, Rucio is a successful collaborative open source project that is rapidly developing into a common standard for scientific data management.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1051/epjconf/202024511006">doi:10.1051/epjconf/202024511006</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/jli5rjrpuzdybchmpa7ijisuqa">fatcat:jli5rjrpuzdybchmpa7ijisuqa</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20210428011052/http://cds.cern.ch/record/2711755/files/ATL-SOFT-PROC-2020-017.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/45/51/4551c3023495ac7df6310d1d7c97804516686dec.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1051/epjconf/202024511006"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="unlock alternate icon" style="background-color: #fb971f;"></i> Publisher / doi.org </button> </a>

Using WLCG data management software to support other communities

Alastair Dewhurst, Catalin Condurache, Tom Byrne, Ian Johnson
<span title="2019-11-04">2019</span> <i title="Zenodo"> Zenodo </i> &nbsp;
The RAL Tier-1 should provide a complete data management solution for these communities and it was therefore decided to setup a Rucio instance to do this.  ...  When the LHC started data taking in 2009 the data rates were unprecedented for the time and forced the WLCG community develop a range of tools for managing their data across many different sites.  ...  Upload Files • Use Rucio CLI Summary • RAL is looking at long term solutions to common data management problems.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.5281/zenodo.3599647">doi:10.5281/zenodo.3599647</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/ltgqmfvgi5hlpjzv75imcyg4r4">fatcat:ltgqmfvgi5hlpjzv75imcyg4r4</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20200207193246/https://zenodo.org/record/3599647/files/CHEP2019_542.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/70/05/7005bc6a7e3bdf4a5c32a4b10bc2a83945dcc6bb.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.5281/zenodo.3599647"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="unlock alternate icon" style="background-color: #fb971f;"></i> zenodo.org </button> </a>

The eXtreme-DataCloud project solutions for data management services in distributed e-infrastructures

Daniele Cesini, Giacinto Donvito, Alessandro Costantini, Fernando Aguilar Gomez, Doina Cristina Duma, Patrick Fuhrmann, Lukasz Dutka, Matthew Viljolen, Serena Battaglia, Vincent Poireau, Luca Dell'Agnello, Oliver Keeble (+9 others)
<span title="">2020</span> <i title="EDP Sciences"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/oduy4e4fvfhovftixyndrlkomi" style="color: black;">EPJ Web of Conferences</a> </i> &nbsp;
The use cases proposed by the different user communities are addressed integrating different data management services ready to manage an increasing volume of data.  ...  generation extreme scale scientific experiments.  ...  on data lifecycle management.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1051/epjconf/202024504010">doi:10.1051/epjconf/202024504010</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/sxhkjwdqlbdj7lpuejiyzizo7m">fatcat:sxhkjwdqlbdj7lpuejiyzizo7m</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20201117041517/https://www.epj-conferences.org/articles/epjconf/pdf/2020/21/epjconf_chep2020_04010.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/22/8d/228d6c9cbcdb83539f289ea203ec3668b1ddf91b.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1051/epjconf/202024504010"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="unlock alternate icon" style="background-color: #fb971f;"></i> Publisher / doi.org </button> </a>

The ESCAPE Data Lake: The machinery behind testing, monitoring and supporting a unified federated storage infrastructure of the exabyte-scale

Rizart Dona, Riccardo Di Maria, on behalf of the ESCAPE project, C. Biscarat, S. Campana, B. Hegner, S. Roiser, C.I. Rovelli, G.A. Stewart
<span title="">2021</span> <i title="EDP Sciences"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/oduy4e4fvfhovftixyndrlkomi" style="color: black;">EPJ Web of Conferences</a> </i> &nbsp;
science projects a unified scalable data management solution for accessing and analyzing large volumes of scientific data.  ...  In this respect, data transfer and management technologies such as Rucio, FTS and GFAL are employed along with monitoring enabling solutions such as Grafana, Elasticsearch and perf- SONAR.  ...  it according to their scientific workflow scenarios.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1051/epjconf/202125102060">doi:10.1051/epjconf/202125102060</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/zvwawt6yqffyjewaisrn7zjkg4">fatcat:zvwawt6yqffyjewaisrn7zjkg4</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20210915014447/https://www.epj-conferences.org/articles/epjconf/pdf/2021/05/epjconf_chep2021_02060.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/71/74/717421e57a78b366fee86aa2c66eaf03a917f3e8.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1051/epjconf/202125102060"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="unlock alternate icon" style="background-color: #fb971f;"></i> Publisher / doi.org </button> </a>

Data Transfer and Network Services management for Domain Science Workflows [article]

Tom Lehman, Xi Yang, Chin Guok, Frank Wuerthwein, Igor Sfiligoi, John Graham, Aashay Arora, Dima Mishin, Diego Davila, Jonathan Guiang, Tom Hutton, Harvey Newman (+1 others)
<span title="2022-03-20">2022</span> <i > arXiv </i> &nbsp; <span class="release-stage" >pre-print</span>
The initial focus is on the Open Science Grid (OSG)/Compact Muon Solenoid (CMS) Large Hadron Collider (LHC) workflows with Rucio/FTS/XRootD based data transfers and the interoperation with the ESnet SENSE  ...  cycle management, and orchestration.  ...  These systems are briefly described below: • RUCIO -provides scientific collaborations with the functionality to organize, manage, and access their data at scale.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2203.08280v2">arXiv:2203.08280v2</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/w52ay7bkbbf3xp3szg4ccx44aq">fatcat:w52ay7bkbbf3xp3szg4ccx44aq</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20220322031501/https://arxiv.org/ftp/arxiv/papers/2203/2203.08280.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/04/59/0459f0737dc2e5c7f22100f9597874f165f0ab78.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2203.08280v2" title="arxiv.org access"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> arxiv.org </button> </a>

The ATLAS Data Carousel Project Status

Mikhail Borodin, Alessandro Di Girolamo, Edward Karavakis, Alexei Klimentov, Tatiana Korchuganova, Mario Lassnig, Tadashi Maeno, Siarhei Padolski, Xin Zhao, on behalf of ATLAS Software and Computing, C. Biscarat, S. Campana (+4 others)
<span title="">2021</span> <i title="EDP Sciences"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/oduy4e4fvfhovftixyndrlkomi" style="color: black;">EPJ Web of Conferences</a> </i> &nbsp;
This amount of data has to be stored, and the corresponding storage system must ensure fast and reliable data delivery for processing by scientific groups distributed all over the world.  ...  State-of-the-art data and workflow management technologies are under active development, and their current status is presented here.  ...  The Data Carousel is the orchestration between the workflow management systems ProdSys2 and PanDA [3] [4] , the distributed data management (DDM) system Rucio [5] , and the tape services.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1051/epjconf/202125102006">doi:10.1051/epjconf/202125102006</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/3h2rwkzi25edfbz3ojy6gj6mt4">fatcat:3h2rwkzi25edfbz3ojy6gj6mt4</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20211010225647/https://www.epj-conferences.org/articles/epjconf/pdf/2021/05/epjconf_chep2021_02006.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/50/21/5021b77140f46006f7ae28160dd6750714ec2baf.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1051/epjconf/202125102006"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="unlock alternate icon" style="background-color: #fb971f;"></i> Publisher / doi.org </button> </a>

The Data Ocean Project

Martin Barisits, Fernando Barreiro, Thomas Beermann, Karan Bhatia, Kaushik De, Arnaud Dubreuil, Johannes Elmsheuser, Alexei Klimentov, Mario Lassnig, Peter Love, Tadashi Maeno, Andrea Manzi (+10 others)
<span title="">2019</span> <i title="EDP Sciences"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/oduy4e4fvfhovftixyndrlkomi" style="color: black;">EPJ Web of Conferences</a> </i> &nbsp;
The following sections then detail the work done in the data management system Rucio and the workflow management systems PanDA and Harvester to interface Google Cloud Platform with the ATLAS distributed  ...  Transparent use of commercial cloud resources for scientific experiments is a hard problem.  ...  Summary and future work We integrated Google Cloud Storage into the ATLAS data management system Rucio, added Google Compute Engine into the ATLAS workflow management system PanDA using Harvester.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1051/epjconf/201921404020">doi:10.1051/epjconf/201921404020</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/bjvgjrgczrezjk247kylpc4v3e">fatcat:bjvgjrgczrezjk247kylpc4v3e</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20200208105059/https://www.epj-conferences.org/articles/epjconf/pdf/2019/19/epjconf_chep2018_04020.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/ae/07/ae0719ec87004faaa7d931302c7fb809429ce278.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1051/epjconf/201921404020"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="unlock alternate icon" style="background-color: #fb971f;"></i> Publisher / doi.org </button> </a>
&laquo; Previous Showing results 1 &mdash; 15 out of 152 results