25,697 Hits in 4.8 sec

Mixed Precision Quantization of ConvNets via Differentiable Neural Architecture Search [article]

Bichen Wu, Yanghan Wang, Peizhao Zhang, Yuandong Tian, Peter Vajda, Kurt Keutzer
2018 arXiv   pre-print
We formulate this problem as a neural architecture search problem and propose a novel differentiable neural architecture search (DNAS) framework to efficiently explore its exponential search space with  ...  Recent work in network quantization has substantially reduced the time and space complexity of neural network inference, enabling their deployment on embedded and mobile devices with limited computational  ...  The idea of super net and stochastic super net is also used in Saxena & Verbeek (2016) ; Veniat & Denoyer (2017) to explore macro architectures of neural nets.  ... 
arXiv:1812.00090v1 fatcat:twg7dl7xjnc7niukbyw2coip5m

Extending LOUPE for K-space Under-sampling Pattern Optimization in Multi-coil MRI [article]

Jinwei Zhang, Hang Zhang, Alan Wang, Qihao Zhang, Mert Sabuncu, Pascal Spincemaille, Thanh D. Nguyen, Yi Wang
2020 arXiv   pre-print
k-space sampling, rather than approximate stochastic k-space sampling of LOUPE during training, was applied together with a straight-through (ST) estimator to estimate the gradient of the threshold operation  ...  in a neural network; thirdly, modified unrolled optimization network, rather than modified U-Net in LOUPE, was used as the reconstruction network in order to reconstruct multi-coil data properly and reduce  ...  Thirdly, approximate stochastic sampling layer was replaced by a binary stochastic sampling layer with Straight-Through (ST) estimator [3] , which was used to avoid zero gradients when back-propagating  ... 
arXiv:2007.14450v1 fatcat:mkwqlddxg5b77geht6io3fulkm

On the approximation of the inverse dynamics of a robotic manipulator by a neural network trained with a stochastic learning algorithm

Enrique Carlos Segura
2013 Inge-Cuc  
Segura, "On the Approximation of the Inverse Dynamics of a Robotic Manipulator by a Neural Network Trained with a Stochastic Learning Algorithm", inge cuc, vol. 9, no. 2, pp. 39-43, 2013.  ...  The neural net has the form for i = 1,2, being and Here we show some results obtained for the case of . Figure 3 shows the evolution of the mean relative error for independent training sets.  ... 
doaj:4745ddfd81e942eabbd159e6d340a73d fatcat:6eotwatv7fdznnbjn2xh6vkauq

Computational promise of simultaneous recurrent network with a stochastic search mechanism

2004 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat No 04CH37541) IJCNN-04  
Successful application of Simultaneous Recurrent Neural networks to static optimization problems, where the training had been achieved through one of a number of deterministic gradient descent algorithms  ...  Two techniques are employed to assess the added value of a potential enhancement through a stochastic search mechanism: one method entails comparison of SRN performance with a stochastic search algorithm  ...  ACKNOWLEDGEMENT Partial Funding provided through United States National Science Foundation (NSF) Grant (ECS 98-00247) for this research project is gratefully acknowledged.  ... 
doi:10.1109/ijcnn.2004.1380969 fatcat:5cvava5w6rddddjodp54zrbimu

An Image Classification Method Based on Deep Neural Network with Energy Model

Yang Yang, Jinbao Duan, Haitao Yu, Zhipeng Gao, Xuesong Qiu
2018 CMES - Computer Modeling in Engineering & Sciences  
Stochastic depth network can successfully address these problems [Huang, Sun, Liu et al. (2016) ], it can greatly increase the nets depth and bring substantial performance than former networks.  ...  In this paper, we propose a new algorithm called stochastic depth networks with deep energy model (SADIE), and the model improves stochastic depth neural network with deep energy model to provide attributes  ...  But deep neural network based on energy model can train the nets as a whole and characterize the data features through hidden layers.  ... 
doi:10.31614/cmes.2018.04249 fatcat:vytorzfmabhktgocq2mhy7wto4

A Selective Overview of Deep Learning [article]

Jianqing Fan, Cong Ma, Yiqiao Zhong
2019 arXiv   pre-print
To answer these questions, we introduce common neural network models (e.g., convolutional neural nets, recurrent neural nets, generative adversarial nets) and training techniques (e.g., stochastic gradient  ...  While neural networks have a long history, recent advances have greatly improved their performance in computer vision, natural language processing, etc.  ...  This minimization is usually done via stochastic gradient descent (SGD).  ... 
arXiv:1904.05526v2 fatcat:6b2nhrmnsnezbowhahkue637fq

Learning to Optimize Neural Nets [article]

Ke Li, Jitendra Malik
2017 arXiv   pre-print
More specifically, we show that an optimization algorithm trained with the proposed method on the problem of training a neural net on MNIST generalizes to the problems of training neural nets on the Toronto  ...  In this paper, we explore learning an optimization algorithm for training shallow neural nets.  ...  neural net.  ... 
arXiv:1703.00441v2 fatcat:dbz5bj57lve33ko2ninpyju4xe

A comparison of two optimization techniques for sequence discriminative training of deep neural networks

George Saon, Hagen Soltau
2014 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)  
We compare two optimization methods for lattice-based sequence discriminative training of neural network acoustic models: distributed Hessian-free (DHF) and stochastic gradient descent (SGD).  ...  Our findings on two different LVCSR tasks suggest that SGD running on a single GPU machine achieves the best accuracy 2.5 times faster than DHF running on multiple non-GPU machines; however, DHF training  ...  The criterion of choice for training neural nets for classification is cross-entropy.  ... 
doi:10.1109/icassp.2014.6854668 dblp:conf/icassp/SaonS14 fatcat:n2pkpf42k5az3jakxgcwrcpc7i

Binary Neural Networks: A Survey

Haotong Qin, Ruihao Gong, Xianglong Liu, Xiao Bai, Jingkuan Song, Nicu Sebe
2020 Pattern Recognition  
We also investigate other practical aspects of binary neural networks such as the hardware-friendly design and the training tricks.  ...  In this paper, we present a comprehensive survey of these algorithms, mainly categorized into the native solutions directly conducting binarization, and the optimized ones using techniques like minimizing  ...  In [61] , the Learned Quantization (LQ-Nets) attempts to minimize quantization error by jointly training neural networks and quantizers in the network.  ... 
doi:10.1016/j.patcog.2020.107281 fatcat:p7ohjigozza5viejq6x7cyf6zi

A Numerical Investigation of the Minimum Width of a Neural Network [article]

Ibrohim Nosirov, Jeffrey M. Hokanson
2019 arXiv   pre-print
Through numerical experiments, we seek to test the lower bounds established by Hanin in 2017.  ...  Neural network width and depth are fundamental aspects of network topology.  ...  For every width, 10 distinct neural networks (depths 1 through 10) were created to numerically test Hanin's result.  ... 
arXiv:1910.13817v1 fatcat:ueciu2ps5bgfjon3fucrzipifa

Page 53 of Geographical Analysis Vol. 28, Issue 1 [page]

1996 Geographical Analysis  
neural net compared to 0.7821 for the gravity model).  ...  This functional form is modified via the adaptive setting of weights by means of the application of a local learning process that minimizes an error function through an iterative statistical scheme.  ... 

The "wake-sleep" algorithm for unsupervised neural networks

G. Hinton, P Dayan, B. Frey, R. Neal
1995 Science  
An unsupervised learning algorithm for a multilayer network of stochastic neurons is described.  ...  Training involved 500 sweeps through the 700 examples. For testing, each net was run 10 times to estimate the expected description length of the image.  ...  A neural net with 16 input units, 8 units in the first hidden layer, and 1 hidden unit in the second hidden layer was trained on produced by the generative model.  ... 
doi:10.1126/science.7761831 pmid:7761831 fatcat:ul2go6tkb5b2vdtgawfhddrd2u

Learning to simulate precipitation with supervised and generative adversarial modelling [article]

Carlos Alberto Gomez Gonzalez
This might change with more careful training (hyperparameter tuning). • The main difference between the supervised and generative models lies in the stochastic nature of the predictions as shown in Fig  ...  The decoder path combines the feature and spatial information through up-convolutions and concatenations with high-resolution features from the encoder. • The V-NET is aimed at modelling volumetric data  ... 
doi:10.6084/m9.figshare.19779964.v1 fatcat:rv65m5gchfaejhsl7rppc3dkzy

SDE-Net: Equipping Deep Neural Networks with Uncertainty Estimates [article]

Lingkai Kong, Jimeng Sun, Chao Zhang
2020 arXiv   pre-print
Based on this perspective, we propose a neural stochastic differential equation model (SDE-Net) which consists of (1) a drift net that controls the system to fit the predictive function; and (2) a diffusion  ...  The Bayesian framework provides a principled way of uncertainty estimation but is often not scalable to modern deep neural nets (DNNs) that have a large number of parameters.  ...  We propose a deep neural net model for uncertainty quantification based on neural stochastic differential equation.  ... 
arXiv:2008.10546v1 fatcat:2tffhdk3ovaoxf4ybbx42pjykq

Learning to simulate precipitation with supervised and generative learning models

Carlos Gomez Gonzalez, Kim Serradell Maronda, Markus Donat
2020 Zenodo  
This might change with more careful training (hyperparameter tuning). • The main difference between the supervised and generative models lies in the stochastic nature of the predictions as shown in Fig  ...  The decoder path combines the feature and spatial information through up-convolutions and concatenations with high-resolution features from the encoder. • The V-NET is aimed at modelling volumetric data  ... 
doi:10.5281/zenodo.4106514 fatcat:etshystsgjasnaaxkhdzycpnby
« Previous Showing results 1 — 15 out of 25,697 results