Filters








108 Hits in 3.4 sec

Highway and Residual Networks learn Unrolled Iterative Estimation [article]

Klaus Greff and Rupesh K. Srivastava and Jürgen Schmidhuber
2017 arXiv   pre-print
We demonstrate that this viewpoint directly leads to the construction of Highway and Residual networks.  ...  The past year saw the introduction of new architectures such as Highway networks and Residual networks which, for the first time, enabled the training of feedforward networks with dozens to hundreds of  ...  ACKNOWLEDGEMENTS The authors wish to thank Faustino Gomez, Bas Steunebrink, Jonathan Masci, Sjoerd van Steenkiste and Christian Osendorfer for their feedback and support.  ... 
arXiv:1612.07771v3 fatcat:ummcvgzif5gz7jvxkq6vptsst4

Improving Gradient Flow with Unrolled Highway Expectation Maximization [article]

Chonghyuk Song, Eunseok Kim, Inwook Shim
2020 arXiv   pre-print
To address this issue, we propose Highway Expectation Maximization Networks (HEMNet), which is comprised of unrolled iterations of the generalized EM (GEM) algorithm based on the Newton-Rahpson method.  ...  and memory costs compared to standard unrolled EM.  ...  SSN (Jampani et al. 2018 ) combine unrolled EM iterations with a neural network to learn task-specific superpixels.  ... 
arXiv:2012.04926v1 fatcat:yauhsyhhrze5hnl5jkvyqsw6qy

IamNN: Iterative and Adaptive Mobile Neural Network for Efficient Image Classification [article]

Sam Leroux, Pavlo Molchanov, Pieter Simoens, Bart Dhoedt, Thomas Breuel, Jan Kautz
2018 arXiv   pre-print
Deep residual networks (ResNets) made a recent breakthrough in deep learning.  ...  The resulting network is much smaller than the original network and can adapt the computational cost to the complexity of the input image.  ...  In this unrolled iterative estimation view successive layers cooperate to compute a single level of representation.  ... 
arXiv:1804.10123v1 fatcat:vkls76zzmjar3mfmufnxo53epi

Neural Proximal Gradient Descent for Compressive Imaging [article]

Morteza Mardani, Qingyun Sun, Shreyas Vasawanala, Vardan Papyan, Hatef Monajemi, John Pauly, David Donoho
2018 arXiv   pre-print
We learn a proximal map that works well with real images based on residual networks.  ...  Our key findings include: 1. a recurrent ResNet with a single residual block unrolled from an iterative algorithm yields an effective proximal which accurately reveals MR image details. 2.  ...  Enhao Gong, and Dr. Joseph Cheng for fruitful discussions and their feedback about the MRI reconstruction and software implementation.  ... 
arXiv:1806.03963v1 fatcat:nd2yiahcd5bprj7lkpwparnsza

CrescendoNet: A New Deep Convolutional Neural Network with Ensemble Behavior

Xiang Zhang, Nishant Vishwamitra, Hongxin Hu, Feng Luo
2018 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA)  
In experiments, CrescendoNet with only 15 layers outperforms almost all networks without residual connections on benchmark datasets, CIFAR10, CIFAR100, and SVHN.  ...  We introduce a new deep convolutional neural network, CrescendoNet, by stacking simple building blocks without residual connections.  ...  They proposed that residual connections have led to unrolled iterative estimation in ResNet.  ... 
doi:10.1109/icmla.2018.00053 dblp:conf/icmla/ZhangVH018 fatcat:3imfobuzjveellpwoq5qnpoudm

CrescendoNet: A Simple Deep Convolutional Neural Network with Ensemble Behavior [article]

Xiang Zhang, Nishant Vishwamitra, Hongxin Hu, Feng Luo
2018 arXiv   pre-print
In experiments, CrescendoNet with only 15 layers outperforms almost all networks without residual connections on benchmark datasets, CIFAR10, CIFAR100, and SVHN.  ...  from the FractalNet that is also a deep convolutional neural network without residual connections.  ...  They proposed that residual connections have led to unrolled iterative estimation in ResNet.  ... 
arXiv:1710.11176v2 fatcat:4v2axciqzngr7m22j65lmswwtu

Bridging the Gaps Between Residual Learning, Recurrent Neural Networks and Visual Cortex [article]

Qianli Liao, Tomaso Poggio
2020 arXiv   pre-print
We discuss relations between Residual Networks (ResNet), Recurrent Neural Networks (RNNs) and the primate visual cortex.  ...  We propose 1) a generalization of both RNN and ResNet architectures and 2) the conjecture that a class of moderately deep RNNs is a biologically-plausible model of the ventral stream in visual cortex.  ...  The number of layers in the unrolled network corresponds to the discrete time iterations of the dynamical system.  ... 
arXiv:1604.03640v2 fatcat:sneoms6cxvdq3dfga7kcgs757a

Improved Stereo Matching with Constant Highway Networks and Reflective Confidence Learning [article]

Amit Shaked, Lior Wolf
2016 arXiv   pre-print
We propose a new highway network architecture for computing the matching cost at each possible disparity, based on multilevel weighted residual shortcuts, trained with a hybrid loss that supports multilevel  ...  The proposed pipeline achieves state of the art accuracy on the largest and most competitive stereo benchmarks, and the learned confidence is shown to outperform all existing alternatives.  ...  building block is weighted by a learned factor λ, and formally defined as: y i+1 = f i+1 (y i ) + λ i+1 · y i (1) In the highway network, the two terms f i+1 (y i ) and y i are weighted by t i+1 and 1  ... 
arXiv:1701.00165v1 fatcat:6h46ztgemnh3vi5f4jid5su5mq

Improved Stereo Matching with Constant Highway Networks and Reflective Confidence Learning

Amit Shaked, Lior Wolf
2017 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)  
We propose a new highway network architecture for computing the matching cost at each possible disparity, based on multilevel weighted residual shortcuts, trained with a hybrid loss that supports multilevel  ...  The proposed pipeline achieves state of the art accuracy on the largest and most competitive stereo benchmarks, and the learned confidence is shown to outperform all existing alternatives.  ...  building block is weighted by a learned factor λ, and formally defined as: y i+1 = f i+1 (y i ) + λ i+1 · y i (1) In the highway network, the two terms f i+1 (y i ) and y i are weighted by t i+1 and 1  ... 
doi:10.1109/cvpr.2017.730 dblp:conf/cvpr/ShakedW17 fatcat:25v3yf2ntbgaxbdk4xweubbywe

Enhancing Precision with an Ensemble Generative Adversarial Network for Steel Surface Defect Detectors (EnsGAN-SDD)

Fityanul Akhyar, Elvin Nur Furqon, Chih-Yang Lin
2022 Sensors  
by learning the feedback from the sequential feature pyramid network.  ...  To solve these problems, we propose incorporating super-resolution technique, sequential feature pyramid network, and boundary localization.  ...  To implement the recursive operation, we unroll it to a sequential network, i.e., ∀i 1, … , S; t 1, … T. 𝛅 𝛂 𝛅 , 𝐱 , where 𝐱 𝛃 𝐱 , 𝛝 𝛅 . ( 9 ) where T is the number of unrolled iterations, and  ... 
doi:10.3390/s22114257 pmid:35684877 pmcid:PMC9185267 fatcat:tnq26abo6rc5xpcjovet25g7ya

SNODE: Spectral Discretization of Neural ODEs for System Identification [article]

Alessio Quaglino, Marco Gallieri, Jonathan Masci, Jan Koutník
2020 arXiv   pre-print
The series coefficients, as well as the network weights, are computed by minimizing the weighted sum of the loss function and the violation of the ODE-Net dynamics.  ...  The problem is solved by coordinate descent that alternately minimizes, with respect to the coefficients and the weights, two unconstrained sub-problems using standard backpropagation and gradient methods  ...  Highway (Srivastava et al., 2015) and residual networks (He et al., 2015) have been studied in (Greff et al., 2016) as unrolled estimators.  ... 
arXiv:1906.07038v2 fatcat:5hspthjo5jgbdacluxtfbsiaga

Coordinated Sum-Rate Maximization in Multicell MU-MIMO with Deep Unrolling [article]

Lukas Schynol, Marius Pesavento
2022 arXiv   pre-print
To obtain more practical solutions, the unrolling/unfolding of traditional iterative algorithms gained significant attention.  ...  Coordinated weighted sum-rate maximization in multicell MIMO networks with intra- and intercell interference and local channel state at the base stations is recognized as an important yet difficult problem  ...  ACKNOWLEDGEMENTS The authors acknowledge the financial support by the Federal Ministry of Education and Research of Germany in the project "Open6GHub" (grant no. 16KISK014).  ... 
arXiv:2202.10371v1 fatcat:idjnmpwj6fad7b53yqnahelhtu

Full Resolution Image Compression with Recurrent Neural Networks [article]

George Toderici, Damien Vincent, Nick Johnston, Sung Jin Hwang, David Minnen, Joel Shor, Michele Covell
2017 arXiv   pre-print
All of our architectures consist of a recurrent neural network (RNN)-based encoder and decoder, a binarizer, and a neural network for entropy coding.  ...  We compare RNN types (LSTM, associative LSTM) and introduce a new hybrid of GRU and ResNet.  ...  Inspired by the core ideas from ResNet [8] and Highway Networks [16] , we can think of GRU as a computation block and pass residual information around the block in order to speed up convergence.  ... 
arXiv:1608.05148v2 fatcat:cbcuh6jwpvbrrhba5wtzv4t4ei

Feedback Networks [article]

Amir R. Zamir, Te-Lin Wu, Lin Sun, William Shen, Jitendra Malik, Silvio Savarese
2017 arXiv   pre-print
We hope this study offers new perspectives in quest for more natural and practical learning models.  ...  We put forth a general feedback based learning architecture with the endpoint results on par or better than existing feedforward networks with the addition of the above advantages.  ...  An example is ResNet [19] , introducing parallel residual connections, as well as hypernetworks [18] , highway networks [53] , stochastic depth [24] , RCNN [37] , GoogLeNet [55] .  ... 
arXiv:1612.09508v3 fatcat:hhvh354vcrg5xecjuqhlp7pexi

Learning to Infer

Joseph Marino, Yisong Yue, Stephan Mandt
2018 International Conference on Learning Representations  
In this paper, we propose iterative inference models, which learn how to optimize a variational lower bound through repeatedly encoding gradients.  ...  Our approach generalizes VAEs under certain conditions, and by viewing VAEs in the context of iterative inference, we provide further insight into several recent empirical findings.  ...  Both encoder and decoder networks in the hierarchical model utilized highway skip connections at each layer at both levels.  ... 
dblp:conf/iclr/MarinoYM18 fatcat:zvvre3cfljf43kcn6zngiikhge
« Previous Showing results 1 — 15 out of 108 results