A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is
Data parallelism has become the de facto standard for training Deep Neural Network on multiple processing units. In this work we propose DC-S3GD, a decentralized (without Parameter Server) stale-synchronous version of the Delay-Compensated Asynchronous Stochastic Gradient Descent (DC-ASGD) algorithm. In our approach, we allow for the overlap of computation and communication, and compensate the inherent error with a first-order correction of the gradients. We prove the effectiveness of ourarXiv:1911.02516v1 fatcat:uev2oh4qjref7kezoxj4bsepxa
more »... ch by training Convolutional Neural Network with large batches and achieving state-of-the-art results.
We propose a genetic algorithm (GA) for hyperparameter optimization of artificial neural networks which includes chromosomal crossover as well as a decoupling of parameters (i.e., weights and biases) from hyperparameters (e.g., learning rate, weight decay, and dropout) during sexual reproduction. Children are produced from three parents; two contributing hyperparameters and one contributing the parameters. Our version of population-based training (PBT) combines traditional gradient-basedarXiv:1901.03900v1 fatcat:kcxcxrd5lraafeyymfu3pnw5om
more »... hes such as stochastic gradient descent (SGD) with our GA to optimize both parameters and hyperparameters across SGD epochs. Our improvements over traditional PBT provide an increased speed of adaptation and a greater ability to shed deleterious genes from the population. Our methods improve final accuracy as well as time to fixed accuracy on a wide range of deep neural network architectures including convolutional neural networks, recurrent neural networks, dense neural networks, and capsule networks.
SmartSim is an open source library dedicated to enabling online analysis and Machine Learning (ML) for traditional High Performance Computing (HPC) simulations. SmartSim provides the ability for simulations written in C, C++, Fortran, and Python to call out to PyTorch, TorchScript, TensorFlow, and any model that supports the ONNX format (i.e. scikit-learn). In addition, the in-transit architecture of SmartSim enables simulation data streaming for online analysis, processing, and training. Indoi:10.5281/zenodo.4986181 fatcat:niu4n2jonjdufljji2ddrhvfjm
more »... s talk we detail the SmartSim architecture and provide benchmarks including online inference and throughput on multiple Cray XC50 supercomputers. We will detail examples including how we used SmartSim to run a 12-member ensemble of global-scale, high-resolution ocean simulations, each spanning 19 compute nodes, all communicating with the same ML architecture at each simulation timestep. Lastly, we will present our plans for open source community involvement, and detail current development directions and research.
Mathematics and Visualization
Lagrangian coherent structures play an important role in the analysis of unsteady vector fields because they represent the timedependent analog to vector field topology. Nowadays, they are often obtained as ridges in the finite-time Lyapunov exponent of the vector field. However, one drawback of this quantity is its very high computational cost because a trajectory needs to be computed for every sample in the space-time domain. A focus of this paper are Lagrangian coherent structures that aredoi:10.1007/978-3-642-15014-2_13 fatcat:qg3uzmuoezfgtikpscy6lhjx6y
more »... lated to predefined regions such as boundaries, i.e. related to flow attachment and flow separation phenomena. It presents an efficient method for computing the finite-time Lyapunov exponent and its height ridges only in these regions, and in particular, grid advection for the efficient computation of time series of the finite-time Lyapunov exponent, exploiting temporal coherence.
We demonstrate the first climate-scale, numerical ocean simulations improved through distributed, online inference of Deep Neural Networks (DNN) using SmartSim. SmartSim is a library dedicated to enabling online analysis and Machine Learning (ML) for traditional HPC simulations. In this paper, we detail the SmartSim architecture and provide benchmarks including online inference with a shared ML model on heterogeneous HPC systems. We demonstrate the capability of SmartSim by using it to run aarXiv:2104.09355v1 fatcat:cw7m6p7kk5et7pd23uiueuip6m
more »... member ensemble of global-scale, high-resolution ocean simulations, each spanning 19 compute nodes, all communicating with the same ML architecture at each simulation timestep. In total, 970 billion inferences are collectively served by running the ensemble for a total of 120 simulated years. Finally, we show our solution is stable over the full duration of the model integrations, and that the inclusion of machine learning has minimal impact on the simulation runtimes.
Identifying the boundary beyond which quantum machines provide a computational advantage over their classical counterparts is a crucial step in charting their usefulness. Gaussian Boson Sampling (GBS), in which photons are measured from a highly entangled Gaussian state, is a leading approach in pursuing quantum advantage. State-of-the-art quantum photonics experiments that, once programmed, run in minutes, would require 600 million years to simulate using the best pre-existing classicalarXiv:2108.01622v1 fatcat:cfedyktg5rc5bfnar2uta7ooi4
more »... hms. Here, we present substantially faster classical GBS simulation methods, including speed and accuracy improvements to the calculation of loop hafnians, the matrix function at the heart of GBS. We test these on a ∼ 100,000 core supercomputer to emulate a range of different GBS experiments with up to 100 modes and up to 92 photons. This reduces the run-time of classically simulating state-of-the-art GBS experiments to several months – a nine orders of magnitude improvement over previous estimates. Finally, we introduce a distribution that is efficient to sample from classically and that passes a variety of GBS validation methods, providing an important adversary for future experiments to test against.
La visualizzazione nell'architettura Alessandro F Nell'architettura, la visualizzazione 3D trova la sua principale applicazione come supporto nella fase di progettazione e nell'esposizione del progetto ...doi:10.5169/seals-134272 fatcat:grxas7lae5hhvehruberdde5vi
Identifying the boundary beyond which quantum machines provide a computational advantage over their classical counterparts is a crucial step in charting their usefulness. Gaussian boson sampling (GBS), in which photons are measured from a highly entangled Gaussian state, is a leading approach in pursuing quantum advantage. State-of-the-art GBS experiments that run in minutes would require 600 million years to simulate using the best preexisting classical algorithms. Here, we present fasterdoi:10.1126/sciadv.abl9236 pmid:35080972 pmcid:PMC8791606 fatcat:7dvtzkszgvfg7jur2kflp6zrze
more »... ical GBS simulation methods, including speed and accuracy improvements to the calculation of loop hafnians. We test these on a ∼100,000-core supercomputer to emulate GBS experiments with up to 100 modes and up to 92 photons. This reduces the simulation time for state-of-the-art GBS experiments to several months, a nine-orders of magnitude improvement over previous estimates. Last, we introduce a distribution that is efficient to sample from classically and that passes a variety of GBS validation methods.
Riela, Serena Riganti Fulginei, Francesco Rigazzi, Giovanni Righi, Maria Clelia Riguidel, Michel Riha, Shannon Rikos, Evangelos Rim, You Seung Rinaldi, Stefano Ringle, Christian M. ... Pereira, Carlos Dias Pereira, Elsa Maria Vaz Pereira, Luiz Perelli, Alessandro Peres, Nuno Miguel Machado Reis Perez Diaz, José Luis Pérez Navarro, Antoni Pérez, Gloria Perez-Morago, Hector Perez-Vidal ...doi:10.3390/app8010133 fatcat:3ueglkn6hbcxlpqyy5rm7kxn5e
Rigazzi, M. Passatore, A. Levis, M. Bertini, M. Fascendini, E. ... De Piccoli, Silvia Gattino, Cristina Mosso Percorsi evolutivi e varietà delle imprese etniche in Italia African cinema as instrument and opportunity in inter-academic cooperation: a project in Ngozi Alessandro ...doi:10.13135/2531-8772/789 fatcat:yfm5ksvaajeodc573xlhrtkvl4
For Titor's Equilibrium: Marino Alge, Gioacchino Noris, Alessandro Rigazzi ; for Sea Blast: Urs Dönni, Martin Seiler, Julian Tschannen; for Toon-Dimension: Peter Bucher, Christian Schulz, Nico Ranieri. ...doi:10.1007/978-3-540-69002-3_4 fatcat:nt6qcwi6nbfqtp6vg4jkygj6yi
Giuseppe Rigazzi in Fiorenza, 402 dal Sig. Venanzio Monaldini in Roma 403 e dal Sig. ... Roberti agli Elementi del Manfredi); da ricordare che con lui Algarotti stampò le sue Rime nel 1733.402 Giuseppe Rigazzi in Fiorenza: il Rigazzi dovrrebbe essere il Giuseppe Rigacci, fiorentino, finanziatore ...doi:10.13128/lea-1824-484x-12961 fatcat:xgbdutdm7zailnersoexvogeju