Filters








63 Hits in 6.1 sec

Beyond Backprop: Online Alternating Minimization with Auxiliary Variables [article]

Anna Choromanska, Benjamin Cowen, Sadhana Kumaravel, Ronny Luss, Mattia Rigotti, Irina Rish, Brian Kingsbury, Paolo DiAchille, Viatcheslav Gurev, Ravi Tejwani, Djallel Bouneffouf
2019 arXiv   pre-print
The main contribution of our work is a novel online (stochastic/mini-batch) alternating minimization (AM) approach for training deep neural networks, together with the first theoretical convergence guarantees  ...  These limitations continue to motivate exploration of alternative training algorithms, including several recently proposed auxiliary-variable methods which break the complex nested objective function into  ...  , performing online alternating minimization (AM) over the network weights and auxiliary activation variables, yielding scalability to arbitrarily large dataset, and applicability on incremental/online  ... 
arXiv:1806.09077v4 fatcat:iz4drb6w35hsjfdq3g46ub4uwi

Decoupled Greedy Learning of CNNs [article]

Eugene Belilovsky, Michael Eickenberg, Edouard Oyallon
2020 arXiv   pre-print
Several alternatives that can alleviate this issue have been proposed.  ...  In this context, we consider a simpler, but more effective, substitute that uses minimal feedback, which we call Decoupled Greedy Learning (DGL).  ...  use auxiliary variables to break the optimization into sub-problems.  ... 
arXiv:1901.08164v4 fatcat:4ozv4luj3ffb5faklsd2aoklae

A simple normative network approximates local non-Hebbian learning in the cortex [article]

Siavash Golkar, David Lipshutz, Yanis Bahroun, Anirvan M. Sengupta, Dmitri B. Chklovskii
2020 arXiv   pre-print
derive novel offline and online optimization algorithms, which we call Bio-RRR.  ...  The online algorithms can be implemented by neural networks whose synaptic learning rules resemble calcium plateau potential dependent plasticity observed in the cortex.  ...  Similarly, the auxiliary variable n is represented by the activity of k interneurons with Q encoded in the weights of synapses connecting n to z (purple nodes on the upper dendritic branch of z) and Q  ... 
arXiv:2010.12660v1 fatcat:ly5hr4mr2zc2tkg2pwddzsowfm

Adaptive Extreme Edge Computing for Wearable Devices

Erika Covi, Elisa Donati, Xiangpeng Liang, David Kappel, Hadi Heidari, Melika Payvand, Wei Wang
2021 Frontiers in Neuroscience  
Wearable devices are a fast-growing technology with impact on personal healthcare for both society and economy.  ...  We additionally investigate the challenges beyond neuromorphic computing hardware, algorithms and devices that could impede enhancement of adaptive edge computing in smart wearable devices.  ...  with online training.  ... 
doi:10.3389/fnins.2021.611300 pmid:34045939 pmcid:PMC8144334 fatcat:5by77im5crcslgt7zj3wulzd5e

Predictive Coding: a Theoretical and Experimental Review [article]

Beren Millidge, Anil Seth, Christopher L Buckley
2022 arXiv   pre-print
Predictive coding offers a potentially unifying account of cortical function -- postulating that the core function of the brain is to minimize prediction errors with respect to a generative model of the  ...  The latent variable µs then follow the dynamics of equation (PC) to minimize prediction errors.  ...  accuracy, while good, is not comparable with artificial neural networks trained with backprop.  ... 
arXiv:2107.12979v2 fatcat:hdo7pyz3gzhvrgmsesfpd4mhyq

SuperSpike: Supervised Learning in Multilayer Spiking Neural Networks

Friedemann Zenke, Surya Ganguli
2018 Neural Computation  
Specifically, we test uniform, symmetric and random feedback, finding that simpler tasks can be solved with any type of feedback, while more complex tasks require symmetric feedback.  ...  We use LIF neurons with current-based synaptic input because they can be alternatively formulated via their integral form (cf. equation 2.4).  ...  All cost function values were computed online as the root mean square from a moving average with 10 s time constant.  ... 
doi:10.1162/neco_a_01086 pmid:29652587 fatcat:ytfksmhbgrdrtgghycbvovmuqu

Applications of the Free Energy Principle to Machine Learning and Neuroscience [article]

Beren Millidge
2021 arXiv   pre-print
We go on to propose novel and simpler algorithms which allow for backprop to be implemented in purely local, biologically plausible computations.  ...  errors, showing how predictive coding can be scaled up and extended to be more biologically plausible, and elucidating its close links with other methods such as Kalman Filtering.  ...  an additional degree of freedom, and that with VFE derived functionals we will obtain divergence terms to be minimized between the posterior and prior for the variable, with a FEEF functional we will  ... 
arXiv:2107.00140v1 fatcat:c6phd65xwfc2rcyq7pnth5a3pq

Bayesian Neural Networks: An Introduction and Survey [chapter]

Ethan Goan, Clinton Fookes
2020 Lecture notes in mathematics  
Treat our parameters of interest ω as a position variable. An auxiliary variable is then introduced to model the momentum v of our current position.  ...  beyond a single layer is essential.  ... 
doi:10.1007/978-3-030-42553-1_3 fatcat:rzkjcf6h3vcarkqstsocfjqpri

Learning structured models for active planning : beyond the Markov paradigm towards adaptable abstractions [article]

Robert Lieck, Universität Stuttgart, Universität Stuttgart
2018
An alternative objective is to instead minimize the entropy of the action values H(Q s | ξ) . (4.12) Minimizing H(Q s | ξ) does not guarantee a minimal value of H(a * | ξ) nor the other way around.  ...  Online Learning with Temporally Extended Features Online learning poses a particular challenge because the data set is changing continuously and new data is highly correlated to previous data as the conditional  ...  where the starred variables Q * , Q * are a shorthand notation to indicate updated/recomputed values of Q and Q after (temporally) adding the transition (s, a) → s * ignoring the reward.  ... 
doi:10.18419/opus-10068 fatcat:nnj7s25bp5e4jgx3jn5i4hybhq

Advancing Neuromorphic Computing With Loihi: A Survey of Results and Outlook

Mike Davies, Andreas Wild, Garrick Orchard, Yulia Sandamirskaya, Gabriel A. Fonseca Guerra, Prasad Joshi, Philipp Plank, Sumedh R. Risbud
2021 Proceedings of the IEEE  
This is now changing with the advent of Intel's Loihi, a neuromorphic research processor designed to support a broad range of spiking neural networks with sufficient scale, performance, and features to  ...  orders of magnitude lower latency and energy compared Manuscript The authors are with to state-of-the-art conventional approaches.  ...  uniquely to source-destination neuron pairs, and synaptic scratch variables called tags, inspired from biological models of reinforcement learning, that serve as auxiliary dynamic state variables associated  ... 
doi:10.1109/jproc.2021.3067593 fatcat:krqdmy3u6jdvfl7btjglek5ag4

Advances in Electron Microscopy with Deep Learning

Jeffrey Ede
2020 Zenodo  
This doctoral thesis covers some of my advances in electron microscopy with deep learning.  ...  This version of my thesis is typeset for online dissemination to improve readability, whereas the thesis submitted to the University of Warwick in support of my application for the degree of Doctor of  ...  In addition, part of the text in section 1.2 is adapted from our earlier work with permission 201 under a Creative Commons Attribution 4.0 73 license.  ... 
doi:10.5281/zenodo.4598227 fatcat:hm2ksetmsvf37adjjefmmbakvq

Advances in Electron Microscopy with Deep Learning

Jeffrey Ede
2020 Zenodo  
This doctoral thesis covers some of my advances in electron microscopy with deep learning.  ...  This copy of my thesis is typeset for online dissemination to improve readability, whereas the thesis submitted to the University of Warwick in support of my application for the degree of Doctor of Philosophy  ...  In addition, part of the text in section 1.2 is adapted from our earlier work with permission 201 under a Creative Commons Attribution 4.0 73 license.  ... 
doi:10.5281/zenodo.4399748 fatcat:63ggmnviczg6vlnqugbnrexsgy

Advances in Electron Microscopy with Deep Learning

Jeffrey Ede
2020 Zenodo  
This doctoral thesis covers some of my advances in electron microscopy with deep learning.  ...  This version of my thesis is typeset for online dissemination to improve readability, whereas the thesis submitted to the University of Warwick in support of my application for the degree of Doctor of  ...  In addition, part of the text in section 1.2 is adapted from our earlier work with permission 201 under a Creative Commons Attribution 4.0 73 license.  ... 
doi:10.5281/zenodo.4591029 fatcat:zn2hvfyupvdwlnvsscdgswayci

Advances in Electron Microscopy with Deep Learning

Jeffrey Ede
2020 Zenodo  
This doctoral thesis covers some of my advances in electron microscopy with deep learning.  ...  This copy of my thesis is typeset for online dissemination to improve readability, whereas the thesis submitted to the University of Warwick in support of my application for the degree of Doctor of Philosophy  ...  In addition, part of the text in section 1.2 is adapted from our earlier work with permission 201 under a Creative Commons Attribution 4.0 73 license.  ... 
doi:10.5281/zenodo.4415407 fatcat:6dejwzzpmfegnfuktrld6zgpiq

Advances in Electron Microscopy with Deep Learning

Jeffrey Ede
2020 Zenodo  
This doctoral thesis covers some of my advances in electron microscopy with deep learning.  ...  This copy of my thesis is typeset for online dissemination to improve readability, whereas the thesis submitted to the University of Warwick in support of my application for the degree of Doctor of Philosophy  ...  In addition, part of the text in section 1.2 is adapted from our earlier work with permission 201 under a Creative Commons Attribution 4.0 73 license.  ... 
doi:10.5281/zenodo.4413249 fatcat:35qbhenysfhvza2roihx52afuy
« Previous Showing results 1 — 15 out of 63 results