Filters








143 Hits in 7.6 sec

Homogeneous Vector Capsules Enable Adaptive Gradient Descent in Convolutional Neural Networks [article]

Adam Byerly, Tatiana Kalganova
2021 arXiv   pre-print
Additionally, the introduction of HVCs enables the use of adaptive gradient descent, reducing the dependence a model's achievable accuracy has on the finely tuned hyperparameters of a non-adaptive optimizer  ...  We present a new way of parameterizing and training capsules that we refer to as homogeneous vector capsules (HVCs).  ...  that gives Adam a momentumlike behavior): 1 √v t + (2) Homogeneous Vector Capsules Enable Adaptive Gradient Descent in Convolutional Neural Networkŝ v t is the bias corrected exponential moving average  ... 
arXiv:1906.08676v2 fatcat:2wkzdws6kzdezgxfshfleuovoa

Homogeneous Vector Capsules Enable Adaptive Gradient Descent in Convolutional Neural Networks

Adam Byerly, Tatiana Kalganova
2021 IEEE Access  
Thus, HVCs enable convolutional neural network researchers to: 1) Use adaptive gradient descent methods when training CNNs without experiencing a generalization gap. 2) Save time and compute cycles searching  ...  We interpret homogeneous vector capsules as performing a similar function, at the output stage of a convolutional neural network, as convolutional layers perform at the input stage.  ...  She has over 20 years of experience in design and implementation of applied Intelligent Systems. VOLUME 4, 2016  ... 
doi:10.1109/access.2021.3066842 fatcat:ylyoibmswjdlbibtov5spw6uty

Recognizing Image Objects by Relational Analysis Using Heterogeneous Superpixels and Deep Convolutional Features [article]

Alex Yang, Charlie T. Veal, Derek T. Anderson, Grant J. Scott
2019 arXiv   pre-print
However, rarely is superpixel segmentation examined within the context of deep convolutional neural network architectures.  ...  The results are compared against a baseline deep neural model, as well as among superpixel capsule networks with a variety of hyperparameter settings.  ...  Training Using stochastic gradient descent with a learning rate of 2 × 10 −5 , various Superpixel Capsule Network models were trained with different superpixel sizes for 120 epochs.  ... 
arXiv:1908.00669v1 fatcat:4tw56g5gabe67gveygrkn7m5yi

No Routing Needed Between Capsules [article]

Adam Byerly, Tatiana Kalganova, Ian Dear
2021 arXiv   pre-print
In our study, we show that a simple convolutional neural network using HVCs performs as well as the prior best performing capsule network on MNIST using 5.5x fewer parameters, 4x fewer training epochs,  ...  By using Homogeneous Vector Capsules (HVCs), which use element-wise multiplication rather than matrix multiplication, the dimensions of the capsules remain unentangled.  ...  In addition to the network design and augmentation strategy, the ability to use an adaptive gradient descent method [6] allowed us to achieve this on consumer hardware (2x NVIDIA GeForce GTX 1080 Tis  ... 
arXiv:2001.09136v6 fatcat:a542fbapxvc7xnms7buq7gdxam

Prototype-based Neural Network Layers: Incorporating Vector Quantization [article]

Sascha Saralajew and Lars Holdijk and Maike Rees and Thomas Villmann
2019 arXiv   pre-print
neural networks and vice versa.  ...  Nevertheless, neural networks are lacking robustness and interpretability. Prototype-based vector quantization methods on the other hand are known for being robust and interpretable.  ...  Since the network is optimized by stochastic gradient descent learning, the statistics over the whole dataset cannot be estimated during run-time.  ... 
arXiv:1812.01214v2 fatcat:x5fv4fk7yfcy5o5hfytcej7jbe

CNN Architectures for Geometric Transformation-Invariant Feature Representation in Computer Vision: A Review

Alhassan Mumuni, Fuseini Mumuni
2021 SN Computer Science  
While convolutional neural networks (CNNs) have inherent representation power that provides a high degree of invariance to geometric image transformations, they are unable to satisfactorily handle nontrivial  ...  This challenge arises naturally in many practical machine vision tasks.  ...  invariant features in convolutional neural networks.  ... 
doi:10.1007/s42979-021-00735-0 fatcat:3zrkaan7dncoja4e32u7jgwo4m

End-To-End Data-Dependent Routing in Multi-Path Neural Networks [article]

Dumindu Tissera, Kasun Vithanage, Rukshan Wijessinghe, Subha Fernando, Ranga Rodrigo
2021 arXiv   pre-print
Our multi-path networks show superior performance to existing widening and adaptive feature extraction, and even ensembles, and deeper networks at similar complexity in the image recognition task.  ...  Neural networks are known to give better performance with increased depth due to their ability to learn more abstract features.  ...  Conventional Convolutional Neural Networks with Parallel Paths In this section, we add parallel paths to conventional convolutional neural networks and compare with conventional network widening, deepening  ... 
arXiv:2107.02450v1 fatcat:6xpkpwd43rfhvfxsezlhzzhja4

Neural Computing [article]

Ayushe Gangal, Peeyush Kumar, Sunita Kumari, Aditya Kumar
2021 arXiv   pre-print
Different types of neural networks discovered so far and applications of some of those neural networks are focused on, apart from their theoretical understanding, the working and core concepts involved  ...  in the applications.  ...  It is based on the addition of structures called capsules to the conventional Convoluted Neural Network (CNN).  ... 
arXiv:2107.02744v1 fatcat:kmfb6j3vcrby3mphgzwo6akho4

Kervolutional Neural Networks

Chen Wang, Jianfei Yang, Lihua Xie, Junsong Yuan
2019 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)  
Convolutional neural networks (CNNs) have enabled the state-of-the-art performance in many computer vision tasks.  ...  Extensive experiments show that kervolutional neural networks (KNN) achieve higher accuracy and faster convergence than baseline CNN.  ...  The stochastic gradient descent (SGD) is adopted with momentum of 0.9. We train the networks for 200 epochs with a mini-batch size of 128.  ... 
doi:10.1109/cvpr.2019.00012 dblp:conf/cvpr/WangYXY19 fatcat:r67upgtf5je3zhbplwxokvli2i

Kervolutional Neural Networks [article]

Chen Wang and Jianfei Yang and Lihua Xie and Junsong Yuan
2020 arXiv   pre-print
Convolutional neural networks (CNNs) have enabled the state-of-the-art performance in many computer vision tasks.  ...  Extensive experiments show that kervolutional neural networks (KNN) achieve higher accuracy and faster convergence than baseline CNN.  ...  The stochastic gradient descent (SGD) is adopted with momentum of 0.9. We train the networks for 200 epochs with a mini-batch size of 128.  ... 
arXiv:1904.03955v2 fatcat:hw552w5vz5d5dkdsr2yodpr3pu

Advances in Electron Microscopy with Deep Learning

Jeffrey Ede
2020 Zenodo  
, uniformly spaced and other fixed sparse scan paths; recurrent neural networks trained to piecewise adapt sparse scan paths to specimens by reinforcement learning; improving signal-to-noise; and conditional  ...  and automatic data clustering by t-distributed stochastic neighbour embedding; adaptive learning rate clipping to stabilize learning; generative adversarial networks for compressed sensing with spiral  ...  In addition, part of the text in section 1.2 is adapted from our earlier work with permission 201 under a Creative Commons Attribution 4.0 73 license.  ... 
doi:10.5281/zenodo.4598227 fatcat:hm2ksetmsvf37adjjefmmbakvq

Advances in Electron Microscopy with Deep Learning

Jeffrey Ede
2020 Zenodo  
, uniformly spaced and other fixed sparse scan paths; recurrent neural networks trained to piecewise adapt sparse scan paths to specimens by reinforcement learning; improving signal-to-noise; and conditional  ...  and automatic data clustering by t-distributed stochastic neighbour embedding; adaptive learning rate clipping to stabilize learning; generative adversarial networks for compressed sensing with spiral  ...  In addition, part of the text in section 1.2 is adapted from our earlier work with permission 201 under a Creative Commons Attribution 4.0 73 license.  ... 
doi:10.5281/zenodo.4591029 fatcat:zn2hvfyupvdwlnvsscdgswayci

Advances in Electron Microscopy with Deep Learning

Jeffrey Ede
2020 Zenodo  
, uniformly spaced and other fixed sparse scan paths; recurrent neural networks trained to piecewise adapt sparse scan paths to specimens by reinforcement learning; improving signal-to-noise; and conditional  ...  and automatic data clustering by t-distributed stochastic neighbour embedding; adaptive learning rate clipping to stabilize learning; generative adversarial networks for compressed sensing with spiral  ...  In addition, part of the text in section 1.2 is adapted from our earlier work with permission 201 under a Creative Commons Attribution 4.0 73 license.  ... 
doi:10.5281/zenodo.4399748 fatcat:63ggmnviczg6vlnqugbnrexsgy

Advances in Electron Microscopy with Deep Learning

Jeffrey Ede
2020 Zenodo  
, uniformly spaced and other fixed sparse scan paths; recurrent neural networks trained to piecewise adapt sparse scan paths to specimens by reinforcement learning; improving signal-to-noise; and conditional  ...  and automatic data clustering by t-distributed stochastic neighbour embedding; adaptive learning rate clipping to stabilize learning; generative adversarial networks for compressed sensing with spiral  ...  In addition, part of the text in section 1.2 is adapted from our earlier work with permission 201 under a Creative Commons Attribution 4.0 73 license.  ... 
doi:10.5281/zenodo.4413249 fatcat:35qbhenysfhvza2roihx52afuy

Advances in Electron Microscopy with Deep Learning

Jeffrey Ede
2020 Zenodo  
, uniformly spaced and other fixed sparse scan paths; recurrent neural networks trained to piecewise adapt sparse scan paths to specimens by reinforcement learning; improving signal-to-noise; and conditional  ...  and automatic data clustering by t-distributed stochastic neighbour embedding; adaptive learning rate clipping to stabilize learning; generative adversarial networks for compressed sensing with spiral  ...  In addition, part of the text in section 1.2 is adapted from our earlier work with permission 201 under a Creative Commons Attribution 4.0 73 license.  ... 
doi:10.5281/zenodo.4429792 fatcat:qs6yuapx4vdbdmwna7ix7nnwty
« Previous Showing results 1 — 15 out of 143 results