IA Scholar Query: Interpreting Write Performance of Supercomputer I/O Systems with Regression Models.
https://scholar.archive.org/
Internet Archive Scholar query results feedeninfo@archive.orgThu, 04 Aug 2022 00:00:00 GMTfatcat-scholarhttps://scholar.archive.org/help1440From Data to Software to Science with the Rubin Observatory LSST
https://scholar.archive.org/work/jz4t7hivl5gz5cefeya3zq4qja
The Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST) dataset will dramatically alter our understanding of the Universe, from the origins of the Solar System to the nature of dark matter and dark energy. Much of this research will depend on the existence of robust, tested, and scalable algorithms, software, and services. Identifying and developing such tools ahead of time has the potential to significantly accelerate the delivery of early science from LSST. Developing these collaboratively, and making them broadly available, can enable more inclusive and equitable collaboration on LSST science. To facilitate such opportunities, a community workshop entitled "From Data to Software to Science with the Rubin Observatory LSST" was organized by the LSST Interdisciplinary Network for Collaboration and Computing (LINCC) and partners, and held at the Flatiron Institute in New York, March 28-30th 2022. The workshop included over 50 in-person attendees invited from over 300 applications. It identified seven key software areas of need: (i) scalable cross-matching and distributed joining of catalogs, (ii) robust photometric redshift determination, (iii) software for determination of selection functions, (iv) frameworks for scalable time-series analyses, (v) services for image access and reprocessing at scale, (vi) object image access (cutouts) and analysis at scale, and (vii) scalable job execution systems. This white paper summarizes the discussions of this workshop. It considers the motivating science use cases, identified cross-cutting algorithms, software, and services, their high-level technical specifications, and the principles of inclusive collaborations needed to develop them. We provide it as a useful roadmap of needs, as well as to spur action and collaboration between groups and individuals looking to develop reusable software for early LSST science.Katelyn Breivik, Andrew J. Connolly, K. E. Saavik Ford, Mario Jurić, Rachel Mandelbaum, Adam A. Miller, Dara Norman, Knut Olsen, William O'Mullane, Adrian Price-Whelan, Timothy Sacco, J. L. Sokoloski, Ashley Villar, Viviana Acquaviva, Tomas Ahumada, Yusra AlSayyad, Catarina S. Alves, Igor Andreoni, Timo Anguita, Henry J. Best, Federica B. Bianco, Rosaria Bonito, Andrew Bradshaw, Colin J. Burke, Andresa Rodrigues de Campos, Matteo Cantiello, Neven Caplar, Colin Orion Chandler, James Chan, Luiz Nicolaci da Costa, Shany Danieli, James R. A. Davenport, Giulio Fabbian, Joshua Fagin, Alexander Gagliano, Christa Gall, Nicolás Garavito Camargo, Eric Gawiser, Suvi Gezari, Andreja Gomboc, Alma X. Gonzalez-Morales, Matthew J. Graham, Julia Gschwend, Leanne P. Guy, Matthew J. Holman, Henry H. Hsieh, Markus Hundertmark, Dragana Ilić, Emille E. O. Ishida, Tomislav Jurkić, Arun Kannawadi, Alekzander Kosakowski, Andjelka B. Kovačević, Jeremy Kubica, François Lanusse, Ilin Lazar, W. Garrett Levine, Xiaolong Li, Jing Lu, Gerardo Juan Manuel Luna, Ashish A. Mahabal, Alex I. Malz, Yao-Yuan Mao, Ilija Medan, Joachim Moeyens, Mladen Nikolić, Robert Nikutta, Matt O'Dowd, Charlotte Olsen, Sarah Pearson, Ilhuiyolitzin Villicana Pedraza, Mark Popinchalk, Luka C. Popović, Tyler A. Pritchard, Bruno C. Quint, Viktor Radović, Fabio Ragosta, Gabriele Riccio, Alexander H. Riley, Agata Rożek, Paula Sánchez-Sáez, Luis M. Sarro, Clare Saunders, Đorđe V. Savić, Samuel Schmidt, Adam Scott, Raphael Shirley, Hayden R. Smotherman, Steven Stetzler, Kate Storey-Fisher, Rachel A. Street, David E. Trilling, Yiannis Tsapras, Sabina Ustamujic, Sjoert van Velzen, José Antonio Vázquez-Mata, Laura Venuti, Samuel Wyatt, Weixiang Yu, Ann Zabludoffwork_jz4t7hivl5gz5cefeya3zq4qjaThu, 04 Aug 2022 00:00:00 GMTOrganic Computing
https://scholar.archive.org/work/homwrqlxqverlbvw7riktlawvy
The demand for product individuality increased enormously in recent years and thus affects directly manufacturers and their employees. Due to the increasing demand for batch-size-one production, every product needs specific manufacturing processes. Usually, employees determine these manufacturing steps with provided product data. This research proposal aims to contribute to the extraction of machining processes from product data and their assignment to suitable machinery. We plan to develop an organic computing system based on artificial intelligence methods to solve these problems by including customer-specific designs and employee expertise. The overall objective is to support employees in manufacturing facilities by simplifying the manufacturer and customer interaction.Stefan-Andreas Böhm, Mischa Ahrens, Chandana Priya Nivarthi, Florian Heidecker, Johannes Büttner, Kristina Dingel, Simon Reichhuber, Zhixin Huang, Yujiang He, Sooraj K. Babu, Tuan Pham, Ferdinand Von Tüllenburg, Ingo Thomsen, Aleksey Koschowoj, Universität Kassel, Sven Tomforde, Christian Krupitzerwork_homwrqlxqverlbvw7riktlawvyMon, 18 Jul 2022 00:00:00 GMTHigh-Resolution CMB Bispectrum Estimator
https://scholar.archive.org/work/3aoxzfjta5es7nrwhavsuuhzzq
The Cosmic Microwave Background (CMB) is one of the most valuable probes of the universe we have today. Anisotropies present in the ancient light contain rich statistical information about the perturbations in the early universe and their subsequent evolution until now. The CMB bispectrum, the Fourier equivalent of the three-point correlation function, allows us to study weak non-Gaussian signatures of the primordial fluctuations. Primordial non-Gaussianity is a key prediction of many physically well-motivated inflation models, and measuring its shape and amplitude allows us to constrain various models of the early universe. This thesis comprises two sections. In the first section, we present forecasts on primordial non-Gaussianity constraints from upcoming CMB surveys. We focus our attention on models favoured by the Planck analysis, where a sharp feature in either the inflationary potential or sound speed causes oscillations in the bispectrum. Using preliminary specifications, we find that the Simons Observatory will have up to a factor of 1.6 improvements over Planck, increased to 1.7-2.2 for the CMB Stage-4 experiment. Motivated by bright prospects, we developed a novel CMB bispectrum estimator suited for the resolution and sensitivity of future surveys. We discuss our high-resolution bispectrum estimator in the second section. Our code, named CMB-BEst, utilises a set of general basis functions to accurately constrain a wide variety of models. Implementing such a flexible and precise estimator was a computationally challenging task. We detail our algorithm design, code optimisation and parallelisation for high-performance computing clusters, which made this challenging computation tractable. Validation tests, both for internal consistency and comparisons against conventional estimators, are provided together with a proof-of-concept application. We highlight how CMB-BEst can be used for both general and targeted analyses of previously unconstrained models.Wu Hyun Sohn, Apollo-University Of Cambridge Repository, James Fergusson, Edward Shellardwork_3aoxzfjta5es7nrwhavsuuhzzqFri, 08 Jul 2022 00:00:00 GMTPerfluoroalkylfullerenes
https://scholar.archive.org/work/hmgmq26fnjec5jnic6vfouqncq
New chemical derivatives that possess the greatest variety of addition patterns than any other class of fullerene derivatives represent an important addition to the existing classes of perfluorocarbons, that is, compounds that are composed only of the two types of atoms, carbon and fluorine. These include aromatic and aliphatic perfluorocarbons such as perfluorodecalin, perfluorononane, hexafluorobenzene, etc., which are important as fluorous solvents used in medicine. The propensity of perfluoroalkylfullerenes (PFAFs) to readily crystallize from organic solutions upon slow evaporation in open air provided a straightforward access to their molecular structures via X-ray crystallography. Another crucial aspect that ensures future success in the characterization of numerous PFAFs of higher fullerenes and endohedral metallofullerenes is the possibility to apply HPLC methodologies to the separation of product mixtures. PFAFs, especially those of C60 and C70, are unique fullerene derivatives in terms of the number of structurally characterized derivatives with different number of RF groups and different addition patterns.Olga V. Boltalina, Alexey A. Popov, Igor V. Kuvychko, Natalia B. Shustova, Steven H. Strauss, Technische Informationsbibliothek (TIB)work_hmgmq26fnjec5jnic6vfouqncqThu, 30 Jun 2022 00:00:00 GMTHeterogeneous Sparse Matrix-Vector Multiplication via Compressed Sparse Row Format
https://scholar.archive.org/work/kju6yzio3rdxjmzmuquvvsewuy
Due to ill performance on many devices, sparse matrix-vector multiplication (SpMV) normally requires special care to store and tune for a given device. However, SpMV is one of the most important kernels in high-performance computing (HPC), and therefore, a storage format and tuning are required that allows for efficient SpMV operations with low memory and tuning overheads across heterogeneous devices. Additionally, the primary users of SpMV operations in HPC are normally application scientists that already have numerous other libraries they depend on the use of some standard sparse matrix storage format. As such, the ideal heterogeneous format would also be something that could easily be understood and requires no major changes to be translated into a standard sparse matrix format, such as compressed sparse row (CSR). This paper presents a heterogeneous format based on CSR, named CSR-k, that can be tuned quickly, requires minimal memory overheads, outperforms the average performance of NVIDIA's cuSPARSE and Sandia National Laboratories' KokkosKernels, while being on par with Intel MKL on our test suite. Additionally, CSR-k does not need any conversion to be used by standard library calls that require a CSR format input. In particular, CSR-k achieves this by grouping rows into a hierarchical structure of super-rows and super-super-rows that are represented by just a few extra arrays of pointers (i.e., <2.5% memory overhead to keep arrays for both GPU and CPU execution). Due to its simplicity, a model can be tuned for a device, and this model can be used to select super-row and super-super-rows sizes in constant time. We observe in this paper that CSR-k can achieve about 17.3% improvement on an NVIDIA V100 and about 18.9% improvement on an NVIDIA A100 over NVIDIA's cuSPARSE while still performing on-par with Intel MKL on an Intel Xeon Platinum 8380 and an AMD Epyc 7742.Phillip Allen Lane, Joshua Dennis Boothwork_kju6yzio3rdxjmzmuquvvsewuyFri, 24 Jun 2022 00:00:00 GMTMolecular Dynamics for Synthetic Biology
https://scholar.archive.org/work/tjd3uiulavgb5bvr5cf6af2ify
Synthetic biology is the field concerned with the design, engineering, and construction of organisms and biomolecules. Biomolecules such as proteins are nature's nano-bots, and provide both a shortcut to the construction of nano-scale tools and insight into the design of abiotic nanotechnology. A fundamental technique in protein engineering is protein fusion, the concatenation of two proteins so that they form domains of a new protein. The resulting fusion protein generally retains both functions, especially when a linker sequence is introduced between the two domains to allow them to fold independently. Fusion proteins can have features absent from all of their components; for example, FRET biosensors are fusion proteins of two fluorescent proteins with a binding domain. When the binding domain forms a complex with a ligand, its dynamics translate the concentration of the ligand to the ratio of fluorescence intensities via FRET. Despite these successes, protein engineering remains laborious and expensive. Computer modelling has the potential to improve the situation by enabling some design work to occur virtually. Synthetic biologists commonly use fast, heuristic structure prediction tools like ROSETTA, I-TASSER and FoldX, despite their inaccuracy. By contrast, molecular dynamics with modern force fields has proven itself accurate, but sampling sufficiently to solve problems accurately and quickly enough to be relevant to experimenters remains challenging. In this thesis, I introduce molecular dynamics to a structural biology audience, and discuss the challenges and theory behind the technique. With this knowledge, I introduce synthetic biology through a review of fluorescent sensors. I then develop a simple computational tool, Rangefinder, for the design of one variety of these sensors, and demonstrate its ability to predict sensor performance experimentally. I demonstrate the importance of the choice of linker with yet another sensor whose performance depends critically thereon. In chapter 6, I investigate the [...]Josh Mitchell, University, The Australian Nationalwork_tjd3uiulavgb5bvr5cf6af2ifyWed, 15 Jun 2022 00:00:00 GMTMemory management in hybrid DRAM/NVM systems using LSTMs
https://scholar.archive.org/work/wc2ipxhmc5b3xafufivn6h7rmq
Με επιφύλαξη παντός δικαιώματος. Απαγορεύεται η αντιγραφή, αποθήκευση και διανομή της παρούσας εργασίας, εξ΄ ολοκλήρου ή τμήματος αυτής, για εμπορικό σκοπό. Επιτρέπεται η ανατύπωση, αποθήκευση και διανομή για σκοπό μη κερδοσκοπικό, εκπαιδευτικής ή ερευνητικής φύσης, υπό την προϋπόθεση να αναφέρεται η πηγή προέλευσης και να διατηρείται το παρόν μήνυμα. Ερωτήματα που αφορούν τη χρήση της εργασίας για κερδοσκοπικό σκοπό πρέπει να απευθύνονται προς τον συγγραφέα. Οι απόψεις και τα συμπεράσματα που περιέχονται σε αυτό το έγγραφο εκφράζουν τον συγγραφέα και δεν πρέπει να ερμηνευθεί ότι αντιπροσωπεύουν τις επίσημες θέσεις του Εθνικού Μετσόβιου Πολυτεχνείου.Konstantinos Stavrakakis, National Technological University Of Athenswork_wc2ipxhmc5b3xafufivn6h7rmqTue, 14 Jun 2022 00:00:00 GMTGaia Data Release 3: Astrophysical parameters inference system (Apsis) I – methods and content overview
https://scholar.archive.org/work/kcvnseudsnbwjckj3x47p55rsu
Gaia Data Release 3 contains a wealth of new data products for the community. Astrophysical parameters are a major component of this release. They were produced by the Astrophysical parameters inference system (Apsis) within the Gaia Data Processing and Analysis Consortium. The aim of this paper is to describe the overall content of the astrophysical parameters in Gaia Data Release 3 and how they were produced. In Apsis we use the mean BP/RP and mean RVS spectra along with astrometry and photometry, and we derive the following parameters: source classification and probabilities for 1.6 billion objects, interstellar medium characterisation and distances for up to 470 million sources, including a 2D total Galactic extinction map, 6 million redshifts of quasar candidates and 1.4 million redshifts of galaxy candidates, along with an analysis of 50 million outlier sources through an unsupervised classification. The astrophysical parameters also include many stellar spectroscopic and evolutionary parameters for up to 470 million sources. These comprise Teff, logg, and m_h (470 million using BP/RP, 6 million using RVS), radius (470 million), mass (140 million), age (120 million), chemical abundances (up to 5 million), diffuse interstellar band analysis (0.5 million), activity indices (2 million), H-alpha equivalent widths (200 million), and further classification of spectral types (220 million) and emission-line stars (50 thousand). This catalogue is the most extensive homogeneous database of astrophysical parameters to date, and it is based uniquely on Gaia data.O.L. Creevey, R. Sordo, F. Pailler, Y. Frémat, U. Heiter, F. Thévenin, R. Andrae, M. Fouesneau, A. Lobel, C.A.L. Bailer-Jones, D. Garabato, I. Bellas-Velidis, E. Brugaletta, A. Lorca, C. Ordenovic, P.A. Palicio, L.M. Sarro, L. Delchambre, R. Drimmel, J. Rybizki, G. Torralba Elipe, A.J. Korn, A. Recio-Blanco, M.S. Schultheis, F. De Angeli, P. Montegriffo, A. Abreu Aramburu, S. Accart, M.A. Álvarez, J. Bakker, N. Brouillet, A. Burlacu, R. Carballo, L. Casamiquela, A. Chiavassa, G. Contursi, W.J. Cooper, C. Dafonte, A. Dapergolas, P. de Laverny, T.E. Dharmawardena, B. Edvardsson, Y. Le Fustec, P. García-Lario, M. García-Torres, A. Gomez, I. González-Santamaría, D. Hatzidimitriou, A. Jean-Antoine Piccolo, M. Kontizas, G. Kordopatis, A.C. Lanzafame, Y. Lebreton, E.L. Licata, H.E.P. Lindstrøm, E. Livanou, A. Magdaleno Romeo, M. Manteiga, F. Marocco, D.J. Marshall, N. Mary, C. Nicolas, L. Pallas-Quintela, C. Panem, B. Pichon, E. Poggio, F. Riclet, C. Robin, R. Santoveña, A. Silvelo, I. Slezak, R.L. Smart, C. Soubiran, M. Süveges, A. Ulla, E. Utrilla, A. Vallenari, H. Zhao, J. Zorec, D. Barrado, A. Bijaoui, J.-C. Bouret, R. Blomme, I. Brott, S. Cassisi, O. Kochukhov, C. Martayan, D. Shulyak, J. Silvesterwork_kcvnseudsnbwjckj3x47p55rsuMon, 13 Jun 2022 00:00:00 GMTEilmer: an Open-Source Multi-Physics Hypersonic Flow Solver
https://scholar.archive.org/work/epusujlplzcpjbcup22x4cizcm
This paper introduces Eilmer, a general-purpose open-source compressible flow solver developed at the University of Queensland, designed to support research calculations in hypersonics and high-speed aerothermodynamics. Eilmer has a broad userbase in several university research groups and a wide range of capabilities, which are documented on the project's website, in the accompanying reference manuals, and in an extensive catalogue of example simulations. The first part of this paper describes the formulation of the code: the equations, physical models, and numerical methods that are used in a basic fluid dynamics simulation, as well as a handful of optional multi-physics models that are commonly added on to do calculations of hypersonic flow. The second section describes the processes used to develop and maintain the code, documenting our adherence to good programming practice and endorsing certain techniques that seem to be particularly helpful for scientific codes. The final section describes a half-dozen example simulations that span the range of Eilmer's capabilities, each consisting of some sample results and a short explanation of the problem being solved, which together will hopefully assist new users in beginning to use Eilmer in their own research projects.Nicholas N. Gibbons and Kyle A. Damm and Peter A. Jacobs and Rowan J. Gollanwork_epusujlplzcpjbcup22x4cizcmFri, 03 Jun 2022 00:00:00 GMTA Study of Failure Recovery and Logging of High-Performance Parallel File Systems
https://scholar.archive.org/work/sm6hajof4relpbed2takkfdaga
Large-scale parallel file systems (PFSs) play an essential role in high-performance computing (HPC). However, despite their importance, their reliability is much less studied or understood compared with that of local storage systems or cloud storage systems. Recent failure incidents at real HPC centers have exposed the latent defects in PFS clusters as well as the urgent need for a systematic analysis. To address the challenge, we perform a study of the failure recovery and logging mechanisms of PFSs in this article. First, to trigger the failure recovery and logging operations of the target PFS, we introduce a black-box fault injection tool called PFault , which is transparent to PFSs and easy to deploy in practice. PFault emulates the failure state of individual storage nodes in the PFS based on a set of pre-defined fault models and enables examining the PFS behavior under fault systematically. Next, we apply PFault to study two widely used PFSs: Lustre and BeeGFS. Our analysis reveals the unique failure recovery and logging patterns of the target PFSs and identifies multiple cases where the PFSs are imperfect in terms of failure handling. For example, Lustre includes a recovery component called LFSCK to detect and fix PFS-level inconsistencies, but we find that LFSCK itself may hang or trigger kernel panics when scanning a corrupted Lustre. Even after the recovery attempt of LFSCK, the subsequent workloads applied to Lustre may still behave abnormally (e.g., hang or report I/O errors). Similar issues have also been observed in BeeGFS and its recovery component BeeGFS-FSCK. We analyze the root causes of the abnormal symptoms observed in depth, which has led to a new patch set to be merged into the coming Lustre release. In addition, we characterize the extensive logs generated in the experiments in detail and identify the unique patterns and limitations of PFSs in terms of failure logging. We hope this study and the resulting tool and dataset can facilitate follow-up research in the communities and help improve PFSs for reliable high-performance computing.Runzhou Han, Om Rameshwar Gatla, Mai Zheng, Jinrui Cao, Di Zhang, Dong Dai, Yong Chen, Jonathan Cookwork_sm6hajof4relpbed2takkfdagaTue, 31 May 2022 00:00:00 GMT2022 Review of Data-Driven Plasma Science
https://scholar.archive.org/work/22ybajli5fa33dpewomz6hi7cu
Data science and technology offer transformative tools and methods to science. This review article highlights latest development and progress in the interdisciplinary field of data-driven plasma science (DDPS). A large amount of data and machine learning algorithms go hand in hand. Most plasma data, whether experimental, observational or computational, are generated or collected by machines today. It is now becoming impractical for humans to analyze all the data manually. Therefore, it is imperative to train machines to analyze and interpret (eventually) such data as intelligently as humans but far more efficiently in quantity. Despite the recent impressive progress in applications of data science to plasma science and technology, the emerging field of DDPS is still in its infancy. Fueled by some of the most challenging problems such as fusion energy, plasma processing of materials, and fundamental understanding of the universe through observable plasma phenomena, it is expected that DDPS continues to benefit significantly from the interdisciplinary marriage between plasma science and data science into the foreseeable future.Rushil Anirudh, Rick Archibald, M. Salman Asif, Markus M. Becker, Sadruddin Benkadda, Peer-Timo Bremer, Rick H.S. Budé, C.S. Chang, Lei Chen, R. M. Churchill, Jonathan Citrin, Jim A Gaffney, Ana Gainaru, Walter Gekelman, Tom Gibbs, Satoshi Hamaguchi, Christian Hill, Kelli Humbird, Sören Jalas, Satoru Kawaguchi, Gon-Ho Kim, Manuel Kirchen, Scott Klasky, John L. Kline, Karl Krushelnick, Bogdan Kustowski, Giovanni Lapenta, Wenting Li, Tammy Ma, Nigel J. Mason, Ali Mesbah, Craig Michoski, Todd Munson, Izumi Murakami, Habib N. Najm, K. Erik J. Olofsson, Seolhye Park, J. Luc Peterson, Michael Probst, Dave Pugmire, Brian Sammuli, Kapil Sawlani, Alexander Scheinker, David P. Schissel, Rob J. Shalloo, Jun Shinagawa, Jaegu Seong, Brian K. Spears, Jonathan Tennyson, Jayaraman Thiagarajan, Catalin M. Ticoş, Jan Trieschmann, Jan van Dijk, Brian Van Essen, Peter Ventzek, Haimin Wang, Jason T. L. Wang, Zhehui Wang, Kristian Wende, Xueqiao Xu, Hiroshi Yamada, Tatsuya Yokoyama, Xinhua Zhangwork_22ybajli5fa33dpewomz6hi7cuTue, 31 May 2022 00:00:00 GMTSnowmass21 Accelerator Modeling Community White Paper
https://scholar.archive.org/work/lrj2g3ud55a5ldsywgulh27jey
After a summary of relevant comments and recommendations from various reports over the last ten years, this paper examines the modeling needs in accelerator physics, from the modeling of single beams and individual accelerator elements, to the realization of virtual twins that replicate all the complexity to model a particle accelerator complex as accurately as possible. We then discuss cutting-edge and emerging computing opportunities, such as advanced algorithms, AI/ML and quantum computing, computational needs in hardware, software performance, portability and scalability, and needs for scalable I/O and in-situ analysis. Considerations of reliability, long-term sustainability, user support and training are considered next, before discussing the benefits of ecosystems with integrated workflows based on standardized input and output, and with integrated frameworks and data repositories developed as a community. Last, we highlight how the community can work more collaboratively and efficiently through the development of consortia and centers, and via collaboration with industry.S. Biedron, L. Brouwer, D.L. Bruhwiler, N. M. Cook, A. L. Edelen, D. Filippetto, C.-K. Huang, A. Huebl, N. Kuklev, R. Lehe, S. Lund, C. Messe, W. Mori, C.-K. Ng, D. Perez, P. Piot, J. Qiang, R. Roussel, D. Sagan, A. Sahai, A. Scheinker, F. Tsung, J.-L. Vay, D. Winklehner, H. Zhangwork_lrj2g3ud55a5ldsywgulh27jeySun, 15 May 2022 00:00:00 GMTHighdicom: A Python library for standardized encoding of image annotations and machine learning model outputs in pathology and radiology
https://scholar.archive.org/work/s3prxgsrc5bw5oeaasqvl72evy
Machine learning is revolutionizing image-based diagnostics in pathology and radiology. ML models have shown promising results in research settings, but their lack of interoperability has been a major barrier for clinical integration and evaluation. The DICOM a standard specifies Information Object Definitions and Services for the representation and communication of digital images and related information, including image-derived annotations and analysis results. However, the complexity of the standard represents an obstacle for its adoption in the ML community and creates a need for software libraries and tools that simplify working with data sets in DICOM format. Here we present the highdicom library, which provides a high-level application programming interface for the Python programming language that abstracts low-level details of the standard and enables encoding and decoding of image-derived information in DICOM format in a few lines of Python code. The highdicom library ties into the extensive Python ecosystem for image processing and machine learning. Simultaneously, by simplifying creation and parsing of DICOM-compliant files, highdicom achieves interoperability with the medical imaging systems that hold the data used to train and run ML models, and ultimately communicate and store model outputs for clinical use. We demonstrate through experiments with slide microscopy and computed tomography imaging, that, by bridging these two ecosystems, highdicom enables developers to train and evaluate state-of-the-art ML models in pathology and radiology while remaining compliant with the DICOM standard and interoperable with clinical systems at all stages. To promote standardization of ML research and streamline the ML model development and deployment process, we made the library available free and open-source.Christopher P. Bridge, Chris Gorman, Steven Pieper, Sean W. Doyle, Jochen K. Lennerz, Jayashree Kalpathy-Cramer, David A. Clunie, Andriy Y. Fedorov, Markus D. Herrmannwork_s3prxgsrc5bw5oeaasqvl72evySun, 08 May 2022 00:00:00 GMTEnabling Dynamic and Intelligent Workflows for HPC, Data Analytics, and AI Convergence
https://scholar.archive.org/work/xmh7kusl4zemzp3dtwmjvkw3ia
The evolution of High-Performance Computing (HPC) platforms enables the design and execution of progressively larger and more complex workflow applications in these systems. The complexity comes not only from the number of elements that compose the workflows but also from the type of computations they perform. While traditional HPC workflows target simulations and modelling of physical phenomena, current needs require in addition data analytics (DA) and artificial intelligence (AI) tasks. However, the development of these workflows is hampered by the lack of proper programming models and environments that support the integration of HPC, DA, and AI, as well as the lack of tools to easily deploy and execute the workflows in HPC systems. To progress in this direction, this paper presents use cases where complex workflows are required and investigates the main issues to be addressed for the HPC/DA/AI convergence. Based on this study, the paper identifies the challenges of a new workflow platform to manage complex workflows. Finally, it proposes a development approach for such a workflow platform addressing these challenges in two directions: first, by defining a software stack that provides the functionalities to manage these complex workflows; and second, by proposing the HPC Workflow as a Service (HPCWaaS) paradigm, which leverages the software stack to facilitate the reusability of complex workflows in federated HPC infrastructures. Proposals presented in this work are subject to study and development as part of the EuroHPC eFlows4HPC project.Jorge Ejarque, Rosa M. Badia, Loïc Albertin, Giovanni Aloisio, Enrico Baglione, Yolanda Becerra, Stefan Boschert, Julian R. Berlin, Alessandro D'Anca, Donatello Elia, François Exertier, Sandro Fiore, José Flich, Arnau Folch, Steven J Gibbons, Nikolay Koldunov, Francesc Lordan, Stefano Lorito, Finn Løvholt, Jorge Macías, Fabrizio Marozzo, Alberto Michelini, Marisol Monterrubio-Velasco, Marta Pienkowska, Josep de la Puente, Anna Queralt, Enrique S. Quintana-Ortí, Juan E. Rodríguez, Fabrizio Romano, Riccardo Rossi, Jedrzej Rybicki, Miroslaw Kupczyk, Jacopo Selva, Domenico Talia, Roberto Tonini, Paolo Trunfio, Manuela Volpwork_xmh7kusl4zemzp3dtwmjvkw3iaWed, 20 Apr 2022 00:00:00 GMTA Taxonomy of Error Sources in HPC I/O Machine Learning Models
https://scholar.archive.org/work/tmkllm72bvckrfynys6eqm6iuu
I/O efficiency is crucial to productivity in scientific computing, but the increasing complexity of the system and the applications makes it difficult for practitioners to understand and optimize I/O behavior at scale. Data-driven machine learning-based I/O throughput models offer a solution: they can be used to identify bottlenecks, automate I/O tuning, or optimize job scheduling with minimal human intervention. Unfortunately, current state-of-the-art I/O models are not robust enough for production use and underperform after being deployed. We analyze multiple years of application, scheduler, and storage system logs on two leadership-class HPC platforms to understand why I/O models underperform in practice. We propose a taxonomy consisting of five categories of I/O modeling errors: poor application and system modeling, inadequate dataset coverage, I/O contention, and I/O noise. We develop litmus tests to quantify each category, allowing researchers to narrow down failure modes, enhance I/O throughput models, and improve future generations of HPC logging and analysis tools.Mihailo Isakov, Mikaela Currier, Eliakin del Rosario, Sandeep Madireddy, Prasanna Balaprakash, Philip Carns, Robert B. Ross, Glenn K. Lockwood, Michel A. Kinsywork_tmkllm72bvckrfynys6eqm6iuuMon, 18 Apr 2022 00:00:00 GMTSimulating fermionic systems on classical and quantum computing devices
https://scholar.archive.org/work/ddu3szbvtzgntb2tyyo2vj5ena
This thesis presents a theoretical study of topics related to the simulation of quantum mechanical systems on classical and quantum computers. A large part of this work focuses on strongly interacting fermionic systems, more precisely, the behavior of electrons in presence of strong magnetic fields. We show how the energy spectrum of a Hamiltonian describing the fractional quantum Hall effect can be computed on a quantum computer and derive a closed form for the Hamiltonian coefficients in second quantization. We then discuss a mean-field method and a multi-reference state approach that allow for an efficient classical computation and an efficient initial state preparation on a quantum computer. The second part of the thesis presents a detailed description on how long-range interacting fermionic systems can be simulated on classical computers using a variational method, introduce an Ansatz which could potentially simplify numerical simulations and give an explicit quantum circuit that shows how the variational state can be used as an initial state and how it can implemented on a quantum computer. In the last part, two novel protocols are presented that generate a variety of prominent many-body operators from two-body interactions and show how these protocols improve over previous construction schemes for a number of important examples. iv Zusammenfassung Diese Arbeit behandelt verschiedene zentrale Probleme theoretischer Natur, welche im Rahmen der Simulation quantenmechanischer Systeme auf klassischen und Quantencomputern auftreten. Ein Großteil dieser Arbeit beschäftigt sich mit stark wechselwirkendenden fermionischen Systemen, genauer gesagt, dem Verhalten von Elektronen innerhalb eines starken Magnetfelds. Es wird dargelegt, wie das Energiespektrum des Quanten-Hall-Effekt-Hamiltonoperators auf einem Quantencomputer berechnet werden kann, und es werden geschlossene Ausdrücke für dessen Hamilton-Koeffizienten in zweiter Quantisierung hergeleitet. Anschließend werden sowohl ein Molekularfeld-als auch ein Multi-Referenz-Ansatz diskutiert, welche eine effiziente Berechnung auf klassischen Rechnern zulassen sowie eine effiziente Implementierung auf Quantencomputern ermöglichen. Der zweite Teil dieser Arbeit erläutert, wie man langreichweitige, wechselwirkende fermionische Systeme mit Hilfe einer neuen Variationsmethode, welche über die Molekularfeldnäherung hinaus geht, auf einem klassischen Computer simulieren kann. Es wird darüber hinaus ein alternativer Ansatz vorgestellt, der Teile dieser Variationsmethode vereinfachen könnte, und gezeigt, wie sich der Ansatz auf einem Quantencomputer realisieren lässt. Im letzten Teil werden zwei neue Methoden vorgestellt, welche es ermöglichen, eine Reihe wichtiger Vielteilchen-Operatoren aus Zweiteilchen-Wechselwirkungen zu erzeugen. Beide Methoden werden durch eine Vielzahl an wichtigen Beispielen veranschaulicht. v 1 A classical (quantum) algorithm is a step-by-step instruction on how to solve a given problem with operations that can run on a classical (quantum) computer. 2 The runtime is measured by the number of elementary operations used by the respective quantum or classical algorithm. For the former, this can be measured in terms of the quantum circuit model, which is just a specific sequence of elementary quantum operations applied to a number of qubits (a qubit is the quantum analogue to a classical bit). All of this and more is detailed in Sections 1.5 and 1.6. 3 Some of the most prominent classical methods include density functional theory (DFT), which exploits the electron density distribution rather than the many electron wave function using a variety of approximations [19] , but fails at describing strongly interacting systems. Another approach based on the wave function representation is the quantum Monte Carlo (QMC) method, but its efficient implementation suffers from the infamous fermionic sign problem, that leads to an exponential increase in the error of the simulation with system size [20] . Another classical algorithm used to find approximate ground states to the many-body problem is density matrix renormalization group (DMRG) [21] , which very successfully describes one-dimensional systems, but has trouble building up enough entanglement to describe most strongly correlated two-and three-dimensional systems. More on the chemistry side, full configuration quantum Monte Carlo (FCIQMC) is an approach based on QMC, that deals with the One can combine Eqs. (1.2.13) and (1.2.16) and obtain the electronic structure Hamiltonian H = T + V . In doing so, we have quietly neglected the fact that matter consists not only of electrons, but also of nuclei. The nuclei masses are however three orders of magnitudes larger than the masses of the electrons and one can assume the electrons to move within a field of fixed nuclei within good approximation, which is known as the Born-Oppenheimer approximation [45, 46] . A precise non-relativistic treatment of matter would include electron-nucleon, nucleon-nucleon, electron-electron interaction as well as single-body electron and nucleon terms. 9 Note that this definition slightly deviates from the definition we use in Chapters 2 and 3. (1.3.10) By comparing the right-hand sides of Eqs. (1.3.9) and (1.3.10), one realizes that equality requires γ to be composed of anti-commuting variables to obtain non-trivial solutions. Fermionic coherent states For fermionic fields, the only physically realizable eigenstate of the fermionic annihilation operator is the vacuum state. Other eigenstates can be constructed only in a formal way and are merely introduced as a means to do analytical computations. One can show that the unitary displacement operator Functions of Grassmann variables f (γ) which do commute with a Grassmann number are called even, those that anti-commute are called odd functions. 12 We will consider a physical density operator ρ which is a positive Hermitian operator of unit trace, i.o.w. ρ must fulfill The expectation value of a fermionic operator X w.r.t. a normalized quantum state ρ is given by X ρ = tr(ρX). i 2 2Nso p,q=1 θp(Γm) pq θq (1.3.54) for some real and anti-symmetric (i.e. skew-symmetric) (2N so × N so )-matrix Γ m , which is called the correlation matrix. 13 Wick's theorem will be discussed in detail in Appendix 3.B. 14 A basic example for Eq. (1.3.56) and p = 2 is given by 16 The time evolution of a density operator ρ is described by the von-Neumann equation For pure states, the von-Neumann equation is equivalent to the Schrödinger equation. A fault-tolerant quantum computer This thesis considers an idealized quantum computer, arguably the most frustrating assumption for anyone who is trying to run an actual experiment. An idealized quantum computer performs state initialization, gates and measurements without any errors or losses and is perfectly isolated from the environment. While this seems to be the somewhat most unrealistic assumption one can make, it turns out that using quantum error-correction one can not only protect stored and transmitted quantum states, but even protect quantum states which dynamically undergo a quantum computation. This is the content of the following theorem, which we state due to its significance for quantum computing. 5. One has to be able to read out the state of a qubit (in e.g. the computational basis) at the end or even in between the computation. Lemma 1.6.2 (Oblivious amplitude amplification). Let W (V ) be a unitary matrix which acts on n + m (n) qubits ant let θ ∈ (0, π/2). For any |ψ , we let W |0 m |ψ = sin(θ) |0 m V |ψ + cos(θ) |Φ ⊥ , (1.6.13) 34 Where efficiently implementable here again refers to its respective quantum circuit scaling at most polynomially in circuit size and depth (time) with system size. 35 When V is unitary, p can be interpreted as a probability, however, if V is not unitary, p can be larger than 1. V = e −iHt/m = e −i j U j /m .Michael Kaicher, Universität Des Saarlandeswork_ddu3szbvtzgntb2tyyo2vj5enaThu, 14 Apr 2022 00:00:00 GMTApplications and Techniques for Fast Machine Learning in Science
https://scholar.archive.org/work/jsedv4ikcrejljqwaiudndn6be
In this community review report, we discuss applications and techniques for fast machine learning (ML) in science—the concept of integrating powerful ML methods into the real-time experimental data processing loop to accelerate scientific discovery. The material for the report builds on two workshops held by the Fast ML for Science community and covers three main areas: applications for fast ML across a number of scientific domains; techniques for training and implementing performant and resource-efficient ML algorithms; and computing architectures, platforms, and technologies for deploying these algorithms. We also present overlapping challenges across the multiple scientific domains where common solutions can be found. This community report is intended to give plenty of examples and inspiration for scientific discovery through integrated and accelerated ML solutions. This is followed by a high-level overview and organization of technical advances, including an abundance of pointers to source material, which can enable these breakthroughs.Allison McCarn Deiana, Nhan Tran, Joshua Agar, Michaela Blott, Giuseppe Di Guglielmo, Javier Duarte, Philip Harris, Scott Hauck, Mia Liu, Mark S. Neubauer, Jennifer Ngadiuba, Seda Ogrenci-Memik, Maurizio Pierini, Thea Aarrestad, Steffen Bähr, Jürgen Becker, Anne-Sophie Berthold, Richard J. Bonventre, Tomás E. Müller Bravo, Markus Diefenthaler, Zhen Dong, Nick Fritzsche, Amir Gholami, Ekaterina Govorkova, Dongning Guo, Kyle J. Hazelwood, Christian Herwig, Babar Khan, Sehoon Kim, Thomas Klijnsma, Yaling Liu, Kin Ho Lo, Tri Nguyen, Gianantonio Pezzullo, Seyedramin Rasoulinezhad, Ryan A. Rivera, Kate Scholberg, Justin Selig, Sougata Sen, Dmitri Strukov, William Tang, Savannah Thais, Kai Lukas Unger, Ricardo Vilalta, Belina von Krosigk, Shen Wang, Thomas K. Warburtonwork_jsedv4ikcrejljqwaiudndn6beTue, 12 Apr 2022 00:00:00 GMTDagstuhl Reports, Volume 11, Issue 10, October 2021, Complete Issue
https://scholar.archive.org/work/3w5nqw2gangnrkuqgfzp32cw4u
Dagstuhl Reports, Volume 11, Issue 10, October 2021, Complete Issuework_3w5nqw2gangnrkuqgfzp32cw4uMon, 11 Apr 2022 00:00:00 GMTDistributed Deep Learning for Remote Sensing Data Interpretation
https://scholar.archive.org/work/ekxconvndzav3osbwtg4ekxozq
As a newly emerging technology, deep learning is a very promising field in big data applications. Remote sensing often involves huge data volumes obtained daily by numerous in-orbit satellites. This makes it a perfect target area for datadriven applications. Nowadays, technological advances in terms of software and hardware have a noticeable impact on Earth observation applications, more specifically in remote sensing techniques and procedures, allowing for the acquisition of data sets with greater quality at higher acquisition ratios. This results in the collection of huge amounts of remotely sensed data, characterized by their large spatial resolution (in terms of the number of pixels per scene), and very high spectral dimensionality, with hundreds or even thousands of spectral bands. As a result, remote sensing instruments on space borne and airborne platforms are now generating data cubes with extremely high dimensionality, imposing several restrictions in terms of both processing runtimes and storage capacity. In this paper, we provide a comprehensive review of the state-of-the-art in deep learning for remote sensing data interpretation, analyzing the strengths and weaknesses of the most widely used techniques in the literature, as well as an exhaustive description of their parallel and distributed implementations (with particular focus on those conducted using cloud computing systems). We also provide quantitative results, offering an assessment of a deep learning technique in a specific case study (source code available: https://github.com/mhaut/cloud-dnn-HSI). The paper concludes with some remarks and hints about future challenges in the application of deep learning techniques to distributed remote sensing data interpretation problems. We emphasize the role of the cloud in providing a powerful architecture that is now able to manage vast amounts of remotely sensed data due to its implementation simplicity, low cost and high efficiency when compared to other parallel and distributed architectures, such as [...]J. M. Haut, M. E. Paoletti, S. Moreno, J. Plaza, J. A. Rico, A. Plazawork_ekxconvndzav3osbwtg4ekxozqTue, 05 Apr 2022 00:00:00 GMTAn Introduction to Quantum Computing for Statisticians and Data Scientists
https://scholar.archive.org/work/sm2v5mh6pnc6tgepm7ing2ljg4
Quantum computers promise to surpass the most powerful classical supercomputers when it comes to solving many critically important practical problems, such as pharmaceutical and fertilizer design, supply chain and traffic optimization, or optimization for machine learning tasks. Because quantum computers function fundamentally differently from classical computers, the emergence of quantum computing technology will lead to a new evolutionary branch of statistical and data analytics methodologies. This review provides an introduction to quantum computing designed to be accessible to statisticians and data scientists, aiming to equip them with an overarching framework of quantum computing, the basic language and building blocks of quantum algorithms, and an overview of existing quantum applications in statistics and data analysis. Our goal is to enable statisticians and data scientists to follow quantum computing literature relevant to their fields, to collaborate with quantum algorithm designers, and, ultimately, to bring forth the next generation of statistical and data analytics tools.Anna Lopatnikova, Minh-Ngoc Tran, Scott A. Sissonwork_sm2v5mh6pnc6tgepm7ing2ljg4Sun, 03 Apr 2022 00:00:00 GMT