Internet Archive Scholar

The IceProd framework: Distributed data processing for the IceCube neutrino observatory

M.G. Aartsen, R. Abbasi, M. Ackermann, J. Adams, J.A. Aguilar, M. Ahlers, D. Altmann, C. Arguelles, J. Auffenberg, X. Bai, M. Baker, S.W. Barwick (+275 others)
2015 Journal of Parallel and Distributed Computing  

Preserved Fulltext

fulltext thumbnail
Web Archive Capture PDF (521.6 kB)
https://web.archive.org/web/20170921213928/http://bib-pubdb1.desy.de/record/166426/files/DESY-2014-01330.pdf

A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL. The file type is application/pdf.

Other Versions

2014 pre-print
arXiv:1311.5904v2 fatcat:7ohktte5g5bw7d6qbzk3gkgwau
2014 accepted
arXiv:1311.5904v3 fatcat:tmkwas2oircrtae35he6kzbtie
2013 pre-print
arXiv:1311.5904v1 fatcat:jwuyt5gk7bbwhch2r5lapcbqs4
IceCube is a one-gigaton instrument located at the geographic South Pole, designed to detect cosmic neutrinos, iden- tify the particle nature of dark matter, and study high-energy neutrinos themselves. Simulation of the IceCube detector and processing of data require a significant amount of computational resources. IceProd is a distributed management system based on Python, XML-RPC and GridFTP. It is driven by a central database in order to coordinate and admin- ister production of simulations
more » ... nd processing of data produced by the IceCube detector. IceProd runs as a separate layer on top of other middleware and can take advantage of a variety of computing resources, including grids and batch systems such as CREAM, Condor, and PBS. This is accomplished by a set of dedicated daemons that process job submission in a coordinated fashion through the use of middleware plugins that serve to abstract the details of job submission and job management from the framework.
doi:10.1016/j.jpdc.2014.08.001 fatcat:3yjku2lezzdebi6yub7cirtfza
Multiple Versions
Citation

M.G. Aartsen, R. Abbasi, M. Ackermann, J. Adams, J.A. Aguilar, M. Ahlers, D. Altmann, C. Arguelles, J. Auffenberg, X. Bai, M. Baker, S.W. Barwick, V. Baum, R. Bay, J.J. Beatty, J. Becker Tjus, K.-H. Becker, S. BenZvi, P. Berghaus, D. Berley, E. Bernardini, A. Bernhard, D.Z. Besson, G. Binder, D. Bindig, M. Bissok, E. Blaufuss, J. Blumenthal, D.J. Boersma, C. Bohm, D. Bose, S. Böser, O. Botner, L. Brayeur, H.-P. Bretz, A.M. Brown, R. Bruijn, J. Casey, M. Casier, D. Chirkin, A. Christov, B. Christy, K. Clark, L. Classen, F. Clevermann, S. Coenders, S. Cohen, D.F. Cowen, A.H. Cruz Silva, M. Danninger, J. Daughhetee, J.C. Davis, M. Day, C. De Clercq, S. De Ridder, P. Desiati, K.D. de Vries, M. de With, T. DeYoung, J.C. Díaz-Vélez, M. Dunkman, R. Eagan, B. Eberhardt, B. Eichmann, J. Eisch, S. Euler, P.A. Evenson, O. Fadiran, A.R. Fazely, A. Fedynitch, J. Feintzeig, T. Feusels, K. Filimonov, C. Finley, T. Fischer-Wasels, S. Flis, A. Franckowiak, K. Frantzen, T. Fuchs, T.K. Gaisser, J. Gallagher, L. Gerhardt, L. Gladstone, T. Glüsenkamp, A. Goldschmidt, G. Golup, J.G. Gonzalez, J.A. Goodman, D. Góra, D.T. Grandmont, D. Grant, P. Gretskov, J.C. Groh, A. Groß, C. Ha, A. Haj Ismail, P. Hallen, A. Hallgren, F. Halzen, K. Hanson, D. Hebecker, D. Heereman, D. Heinen, K. Helbing, R. Hellauer, S. Hickford, G.C. Hill, K.D. Hoffman, R. Hoffmann, A. Homeier, K. Hoshina, F. Huang, W. Huelsnitz, P.O. Hulth, K. Hultqvist, S. Hussain, A. Ishihara, E. Jacobi, J. Jacobsen, K. Jagielski, G.S. Japaridze, K. Jero, O. Jlelati, B. Kaminsky, A. Kappes, T. Karg, A. Karle, M. Kauer, J.L. Kelley, J. Kiryluk, J. Kläs, S.R. Klein, J.-H. Köhne, G. Kohnen, H. Kolanoski, L. Köpke, C. Kopper, S. Kopper, D.J. Koskinen, M. Kowalski, M. Krasberg, A. Kriesten, K. Krings, G. Kroll, J. Kunnen, N. Kurahashi, T. Kuwabara, M. Labare, H. Landsman, M.J. Larson, M. Lesiak-Bzdak, M. Leuermann, J. Leute, J. Lünemann, O. Macías, J. Madsen, G. Maggi, R. Maruyama, K. Mase, H.S. Matis, F. McNally, K. Meagher, M. Merck, G. Merino, T. Meures, S. Miarecki, E. Middell, N. Milke, J. Miller, L. Mohrmann, T. Montaruli, R. Morse, R. Nahnhauer, U. Naumann, H. Niederhausen, S.C. Nowicki, D.R. Nygren, A. Obertacke, S. Odrowski, A. Olivas, A. Omairat, A. O'Murchadha, L. Paul, J.A. Pepper, C. Pérez de los Heros, C. Pfendner, D. Pieloth, E. Pinat, J. Posselt, P.B. Price, G.T. Przybylski, M. Quinnan, L. Rädel, I. Rae, M. Rameez, K. Rawlins, P. Redl, R. Reimann, E. Resconi, W. Rhode, M. Ribordy, M. Richman, B. Riedel, J.P. Rodrigues, C. Rott, T. Ruhe, B. Ruzybayev, D. Ryckbosch, S.M. Saba, H.-G. Sander, M. Santander, S. Sarkar, K. Schatto, F. Scheriau, T. Schmidt, M. Schmitz, S. Schoenen, S. Schöneberg, A. Schönwald, A. Schukraft, L. Schulte, D. Schultz, O. Schulz, D. Seckel, Y. Sestayo, S. Seunarine, R. Shanidze, C. Sheremata, M.W.E. Smith, D. Soldin, G.M. Spiczak, C. Spiering, M. Stamatikos, T. Stanev, N.A. Stanisha, A. Stasik, T. Stezelberger, R.G. Stokstad, A. Stößl, E.A. Strahler, R. Ström, N.L. Strotjohann, G.W. Sullivan, H. Taavola, I. Taboada, A. Tamburro, A. Tepe, S. Ter-Antonyan, G. Tešić, S. Tilav, P.A. Toale, M.N. Tobin, S. Toscano, M. Tselengidou, E. Unger, M. Usner, S. Vallecorsa, N. van Eijndhoven, A. Van Overloop, J. van Santen, M. Vehring, M. Voge, M. Vraeghe, C. Walck, T. Waldenmaier, M. Wallraff, Ch. Weaver, M. Wellons, C. Wendt, S. Westerhoff, N. Whitehorn, K. Wiebe, C.H. Wiebusch, D.R. Williams, H. Wissing, M. Wolf, T.R. Wood, K. Woschnagg, D.L. Xu, X.W. Xu, J.P. Yanez, G. Yodh, S. Yoshida, P. Zarzhitsky, J. Ziemann, S. Zierke, M. Zoll. "The IceProd framework: Distributed data processing for the IceCube neutrino observatory." Journal of Parallel and Distributed Computing (2015) 198-211