Filters








3,797 Hits in 7.8 sec

Simple, Fast, and Flexible Framework for Matrix Completion with Infinite Width Neural Networks [article]

Adityanarayanan Radhakrishnan, George Stefanakis, Mikhail Belkin, Caroline Uhler
2022 arXiv   pre-print
In this work, we develop an infinite width neural network framework for matrix completion that is simple, fast, and flexible.  ...  In particular, we derive the NTK for fully connected and convolutional neural networks for matrix completion.  ...  Discussion In this work, we presented a simple, fast, and flexible framework for matrix completion using the infinite width limit of neural networks, i.e. the neural tangent kernel (NTK).  ... 
arXiv:2108.00131v2 fatcat:7v27livogzenxdhjyfksxmlbeu

Fast Adaptation with Linearized Neural Networks [article]

Wesley J. Maddox, Shuai Tang, Pablo Garcia Moreno, Andrew Gordon Wilson, Andreas Damianou
2021 arXiv   pre-print
Our experiments on both image classification and regression demonstrate the promise and convenience of this framework for transfer learning, compared to neural network fine-tuning.  ...  The inductive biases of trained neural networks are difficult to understand and, consequently, to adapt to new settings.  ...  In this paper, we apply these ideas primarily for fast, convenient, and analytic transfer learning with neural networks.  ... 
arXiv:2103.01439v2 fatcat:cazvznfrufgnlmvm56z5bgwwbu

On Training and Evaluation of Neural Network Approaches for Model Predictive Control [article]

Rebecka Winqvist, Arun Venkitaraman, Bo Wahlberg
2020 arXiv   pre-print
The contribution of this paper is a framework for training and evaluation of Model Predictive Control (MPC) implemented using constrained neural networks.  ...  The motivation is to replace real-time optimization in safety critical feedback control systems with learnt mappings in the form of neural networks with optimization layers.  ...  CONCLUSION We have presented a a framework for off-line training and evaluation of neural networks approaches for MPC.  ... 
arXiv:2005.04112v1 fatcat:nvxjf63xefa7jmjxltqhkujsvy

A theory of representation learning in deep neural networks gives a deep generalisation of kernel methods [article]

Adam X. Yang, Maxime Robeyns, Edward Milsom, Nandi Schoots, Laurence Aitchison
2022 arXiv   pre-print
We therefore develop a new infinite width limit, the representation learning limit, that exhibits representation learning mirroring that in finite-width networks, yet at the same time, remains extremely  ...  Finally, we use this limit and objective to develop a flexible, deep generalisation of kernel methods, that we call deep kernel machines (DKMs).  ...  This contrasts with a deep linear neural network, which has infinitely many optimal settings for the weights. Note that for the objective to be well-defined, we need K(G) to be full-rank.  ... 
arXiv:2108.13097v4 fatcat:jtnn3ftdifgqnfyo6em4l2ckze

A Theory of Neural Tangent Kernel Alignment and Its Influence on Training [article]

Haozhe Shan, Blake Bordelon
2022 arXiv   pre-print
The training dynamics and generalization properties of neural networks (NN) can be precisely characterized in function space via the neural tangent kernel (NTK).  ...  In nonlinear networks with multiple outputs, we identify the phenomenon of kernel specialization, where the kernel function for each output head preferentially aligns to its own target function.  ...  Both evaluation of the infinite width kernels and training were performed with the Neural Tangents API (Novak et al., 2020) . E.1.  ... 
arXiv:2105.14301v2 fatcat:dalsckgfznalnb563j4dn7olvy

Building powerful and equivariant graph neural networks with structural message-passing [article]

Clement Vignac, Andreas Loukas, Pascal Frossard
2020 arXiv   pre-print
Message-passing has proved to be an effective way to design graph neural networks, as it is able to leverage both permutation equivariance and an inductive bias towards learning local structures in order  ...  Second, we propose methods for the parametrization of the message and update functions that ensure permutation equivariance.  ...  Acknowledgments and Disclosure of Funding Clément Vignac would like to thank the Swiss Data Science Center for supporting him through the PhD fellowship program (grant P18-11).  ... 
arXiv:2006.15107v3 fatcat:m76loemdefhlde4gavp6a4xisq

Deep D-bar: Real time Electrical Impedance Tomography Imaging with Deep Neural Networks

S. J. Hamilton, A. Hauptmann
2018 IEEE Transactions on Medical Imaging  
Convolutional Neural Networks provide a powerful framework for post-processing such convolved direct reconstructions.  ...  Results for absolute EIT images are presented using experimental EIT data from the ACT4 and KIT4 EIT systems.  ...  They would like to thank the Isaac Newton Institute for Mathematical Sciences, Cambridge, for support and hospitality during the programme 'Variational methods and effective algorithms for imaging and  ... 
doi:10.1109/tmi.2018.2828303 pmid:29994023 fatcat:priamh4ly5dfhlxo7zezegtwma

Information Flow in Deep Neural Networks [article]

Ravid Shwartz-Ziv
2022 arXiv   pre-print
In our study, we obtained tractable computations of many information-theoretic quantities and their bounds for infinite ensembles of infinitely wide neural networks.  ...  An analytical framework reveals the underlying structure and optimal representations, and a variational framework using deep neural network optimization validates the results.  ...  His immense knowledge and plentiful experience have encouraged me in all the time of my academic research and daily life. He was a remarkable man who gave me so much, and I learned a lot from him.  ... 
arXiv:2202.06749v2 fatcat:eo3pcousavg3zp5xza57kejjq4

Priors in Bayesian Deep Learning: A Review [article]

Vincent Fortuin
2022 arXiv   pre-print
, and Bayesian neural networks.  ...  We hope to motivate practitioners in Bayesian deep learning to think more carefully about the prior specification for their models and to provide them with some inspiration in this regard.  ...  We thank Alex Immer, Adrià Garriga-Alonso, and Claire Vernade for helpful feedback on the draft and Arnold Weber for constant inspiration.  ... 
arXiv:2105.06868v3 fatcat:dmra3u2ibzgrnblzsepjgrr6pm

Structured Low-Rank Algorithms: Theory, MR Applications, and Links to Machine Learning [article]

Mathews Jacob, Merry P. Mani, Jong Chul Ye
2019 arXiv   pre-print
This property enables the reformulation of the signal recovery as a low-rank structured matrix completion, which comes with performance guarantees.  ...  This framework is centered on the fundamental duality between the compactness (e.g., sparsity) of the continuous signal and the rank of a structured matrix, whose entries are functions of the signal.  ...  SLR RECOVERY: FAST ALGORITHMS We will explain the algorithms in the context of a simple Hankel matrix lifting.  ... 
arXiv:1910.12162v1 fatcat:e7x2qzy52fhlbh2rrtasxkc5fi

Deep neural networks with dependent weights: Gaussian Process mixture limit, heavy tails, sparsity and compressibility [article]

Hoil Lee, Fadhel Ayed, Paul Jung, Juho Lee, Hongseok Yang, François Caron
2022 arXiv   pre-print
Under this model, we show that each layer of the infinite-width neural network can be characterised by two simple quantities: a non-negative scalar parameter and a L\'evy measure on the positive reals.  ...  This article studies the infinite-width limit of deep feedforward neural networks whose weights are dependent, and modelled via a mixture of Gaussian distributions.  ...  These neural network models therefore are asymptotically equivalent to a model with iid Gaussian weights in the infinite-width limit.  ... 
arXiv:2205.08187v1 fatcat:vuryqmmdendzfhneuqlqarvg2m

The M2DC Project: Modular Microserver DataCentre

Mariano Cecowski, Giovanni Agosta, Ariel Oleksiak, Michal Kierzynka, Micha vor dem Berge, Wolfgang Christmann, Stefan Krupop, Mario Porrmann, Jens Hagemeyer, Rene Griessl, Meysam Peykanu, Lennart Tigges (+13 others)
2016 2016 Euromicro Conference on Digital System Design (DSD)  
The key advantage is the combination of software-like flexibility with the performance otherwise common to hardware.  ...  This paper provides an overview of the different topics FPGAs have been used for in the last 15 years of research and why they have been chosen over other processing units like e.g. CPUs.  ...  allows for an highly flexible system design.  ... 
doi:10.1109/dsd.2016.76 dblp:conf/dsd/CecowskiAOKBCKP16 fatcat:bu4nbkqaejebjafrotibui6mkq

Applying visual domain style transfer and texture synthesis techniques to audio: insights and challenges

Muhammad Huzaifah bin Md Shahrin, Lonce Wyse
2019 Neural computing & applications (Print)  
Style transfer is a technique for combining two images based on the activations and feature statistics in a deep learning neural network architecture.  ...  Despite the awkward fit, experiments show that the Gram matrix determined "style" for audio is more closely aligned with timbral signatures without temporal structure whereas network layer activity determining  ...  [13] provides precise definitions of content and style for use in the neural network models that produce images that correlate remarkably well with our intuitive understanding of these terms.  ... 
doi:10.1007/s00521-019-04053-8 fatcat:yeq2q37rgnbfhd2sy2333dsi3m

Modularizing Deep Learning via Pairwise Learning With Kernels [article]

Shiyu Duan, Shujian Yu, Jose Principe
2020 arXiv   pre-print
Based on this construction, we then propose a provably optimal modular learning framework for classification that does not require between-module backpropagation.  ...  By redefining the conventional notions of layers, we present an alternative view on finitely wide, fully trainable deep neural networks as stacked linear models in feature spaces, leading to a kernel machine  ...  assume random networks or require infinite widths.  ... 
arXiv:2005.05541v2 fatcat:d4ot6n3evrbq3anjjycgt6uipm

Applying Deep Learning to Fast Radio Burst Classification

Liam Connor, Joeri van Leeuwen
2018 Astronomical Journal  
We construct a tree-like deep neural network (DNN) that takes multiple or individual data products as input (e.g. dynamic spectra and multi-beam detection information) and trains on them simultaneously  ...  We apply deep learning to single pulse classification and develop a hierarchical framework for ranking events by their probability of being true astrophysical transients.  ...  We also thank Jorn Peters, Yunfan (Gerry) Zhang, and Folkert Huizinga for useful discussions.  ... 
doi:10.3847/1538-3881/aae649 fatcat:pcq3h6e6jvg25gd7bjur3sqmkm
« Previous Showing results 1 — 15 out of 3,797 results