6,657 Hits in 5.4 sec

Understanding the difficulty of training deep feedforward neural networks

Xavier Glorot, Yoshua Bengio
2010 Journal of machine learning research  
Our objective here is to understand better why standard gradient descent from random initialization is doing so poorly with deep neural networks, to better understand these recent relative successes and  ...  Surprisingly, we find that saturated units can move out of saturation by themselves, albeit slowly, and explaining the plateaus sometimes seen when training neural networks.  ...  to train feedforward neural networks (Rumelhart et al., 1986) .  ... 
dblp:journals/jmlr/GlorotB10 fatcat:qfrlj2iewfbazhz5pi3dyorpfa

Learning with Collaborative Neural Network Group by Reflection [article]

Liyao Gao, Zehua Cheng
2019 arXiv   pre-print
We can reduce the error rate by 74.5% and reached the accuracy of 99.45% in MNIST with three feedforward networks (4 layers) in one training epoch.  ...  In this paper, we might want to present the Collaborative Neural Network Group (CNNG).  ...  The desired k here should try to minimize the difficulty of the training of task classifier and specialist neural networks.  ... 
arXiv:1901.02433v2 fatcat:d6rf45fptrdihjqc4t3gba3yqm

Deep neural networks: a new framework for modelling biological vision and brain information processing [article]

Nikolaus Kriegeskorte
2015 bioRxiv   pre-print
Convolutional feedforward networks, which now dominate computer vision, take further inspiration from the architecture of the primate visual hierarchy.  ...  Artificial neural networks are inspired by the brain and their computations could be implemented in biological neurons.  ...  FEEDFORWARD NEURAL NETWORKS FOR VISUAL OBJECT RECOGNITION Computer vision has recently come to be dominated by a particular type of deep neural network: the deep feedforward convolutional net.  ... 
doi:10.1101/029876 fatcat:lxuwpdhzrvhpdmtyzg33ogwncy

Employment Data Screening and Destination Prediction of College Students Based on Deep Learning

Ping Cheng, Xin Ning
2022 Wireless Communications and Mobile Computing  
Firstly, based on deep learning and Feedforward Neural Network technology, a prediction model of college students' employment destination with six influencing factors is established, and the prediction  ...  According to the characteristics of college students' employment data, this paper uses the deep-seated neural network with strong learning ability and adaptability to predict college students' employment  ...  Training Process Analysis. The deep Feedforward Neural Network is applied to classify the employment of college graduates.  ... 
doi:10.1155/2022/7173771 fatcat:tejcah6it5b3peuemivi5c47xq

Page 842 of Neural Computation Vol. 6, Issue 5 [page]

1994 Neural Computation  
This result does not give much hope for the existence of an efficient algorithm for loading deep networks. 1 The Relevance of the Loading Problem Research of the neural network models for the computational  ...  Communicated by Stephen Judd Loading Deep Networks Is Hard Jiti Sima Institute of Computer Science, Academy of Sciences of the Czech Republic, Pod voddrenskou vézi 2, 182 07 Praha 8, Czech Republic The  ... 

An Approach to Silhouette Identification Realization for Fashion Images Using Deep Neural Networks

Tetsuo TSURU, Masahiro SUGAHARA, Haruhiko NISHIMURA
2019 Transactions of Japan Society of Kansei Engineering  
Therefore, silhouette learning of clothing by Neural Network Console needs to meet the personal professional styling knowledge.  ...  The objective of our studies is the inquiry into data set of the silhouette styling by machine learning for the luxury brands. The computer vision meets the visual fashion for the long years.  ...  Understanding the difficulty of training deep feedforward neural networks, International Conference on Artificial Intelligence and Statistics, 2010. 13 Kingma, D.  ... 
doi:10.5057/jjske.tjske-d-19-00026 fatcat:jfgwyiczkrcezpcv3qi6vinvqe

Periodic Spectral Ergodicity: A Complexity Measure for Deep Neural Networks and Neural Architecture Search [article]

Mehmet Süzen, J.J. Cerdà, Cornelius Weber
2020 arXiv   pre-print
Establishing associations between the structure and the generalisation ability of deep neural networks (DNNs) is a challenging task in modern machine learning.  ...  Based on this measure a technique is devised to quantify the complexity of deep neural networks from the learned weights and traversing the network connectivity in a sequential manner, hence the term cascading  ...  Acknowledgements and Notes We would like to express our gratitute to Charles Martin for pointing us out the usage of pre-trained weights from pytorchvision and thank the PyTorch core team [24] for bundling  ... 
arXiv:1911.07831v4 fatcat:nh6lfc6lcbfmpfv5tqoctepnvi

Deep Learning in Speech Recognition
音声認識におけるDeep Learningの活用

Ken-ichi Iso
2017 The Brain & Neural Networks  
Systems), pp.501-510. 7) Bourlard, H., Morgan, N. (1993): Connectionist Speech Recognition: A Hybrid Approach, difficulty of training deep feedforward neural networks, ICAIS, pp.249-256. 10) Wiesler,  ...  , A., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., Kingsbury, B. (2012): Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups, IEEE  ... 
doi:10.3902/jnns.24.27 fatcat:2ioqodsou5fhvnwmyi3kj2iosu

Artificial neural networks for neuroscientists: A primer [article]

Guangyu Robert Yang, Xiao-Jing Wang
2020 Neuron   accepted
Artificial neural networks (ANNs) are essential tools in machine learning that have drawn increasing attention in neuroscience.  ...  Then, with a focus on bringing this mathematical framework closer to neurobiology, we detail how to customize the analysis, structure, and learning of ANNs to better address a wide range of challenges  ...  Acknowledgments: We thank Vishwa Goudar and Jacob Portes for helpful comments on a draft of this paper.  ... 
doi:10.1016/j.neuron.2020.09.005 pmid:32970997 arXiv:2006.01001v2 fatcat:jucqkw2kufaerabonik6udtbfi

Deep Molecular Dreaming: Inverse machine learning for de-novo molecular design and interpretability with surjective representations [article]

Cynthia Shen, Mario Krenn, Sagi Eppel, Alan Aspuru-Guzik
2020 arXiv   pre-print
PASITHEA exploits the use of gradients by directly reversing the learning process of a neural network, which is trained to predict real-valued chemical properties.  ...  A striking property of inceptionism is that we can directly probe the model's understanding of the chemical space it was trained on.  ...  As a result of mixed vector representations in training, the model has difficulty in converging.  ... 
arXiv:2012.09712v1 fatcat:7awdssft3nev7hta2msqnqg23u

Multi-head or Single-head? An Empirical Comparison for Transformer Training [article]

Liyuan Liu and Jialu Liu and Jiawei Han
2021 arXiv   pre-print
Meanwhile, we show that, with recent advances in deep learning, we can successfully stabilize the training of the 384-layer Transformer.  ...  Then, we suggest the main advantage of the multi-head attention is the training stability, since it has less number of layers than the single-head attention, when attending the same number of positions  ...  depth can increase model capacity at the cost of training difficulty.  ... 
arXiv:2106.09650v1 fatcat:qrjt2eujdvam3j6oasijluoi4u

Quantum neuromorphic hardware for quantum artificial intelligence

Enrico Prati
2017 Journal of Physics, Conference Series  
The development of machine learning methods based on deep learning boosted the field of artificial intelligence towards unprecedented achievements and application in several fields.  ...  Here I review the convergence between the two fields towards implementation of advanced quantum algorithms, including quantum deep learning.  ...  From machine learning to quantum artificial intelligence In the years, three main methods have been developed to train both feedforward and recurrent neural networks: supervised learning (the training  ... 
doi:10.1088/1742-6596/880/1/012018 fatcat:ivfn67ojgngixgfwvwiq4eih2e

Deep Learning of Representations [chapter]

Yoshua Bengio, Aaron Courville
2013 Intelligent Systems Reference Library  
Understanding the difficulty of training deep feedforward neural networks. In AISTATS'2010. Glorot, X., Bordes, A., and Bengio, Y. (2011).  ...  Practical recommendations for gradient-based training of deep architectures. In K.-R. Müller, G. Montavon, and G. B. Orr, editors, Neural Networks: Tricks of the Trade. Springer.  ... 
doi:10.1007/978-3-642-36657-4_1 fatcat:itdt75pd6bavve25ta4uxruxs4

Underwater Inherent Optical Properties Estimation Using a Depth Aided Deep Neural Network

Zhibin Yu, Yubo Wang, Bing Zheng, Haiyong Zheng, Nan Wang, Zhaorui Gu
2017 Computational Intelligence and Neuroscience  
The power of artificial neural network has been proved in image processing and computer vision fields with deep learning technology.  ...  To overcome this problem, we aim to develop a novel measuring method using only a single underwater image with the help of deep artificial neural network.  ...  Acknowledgments This work was supported by the National Natural Science  ... 
doi:10.1155/2017/8351232 pmid:29270196 pmcid:PMC5706080 fatcat:bjicchmq4bapbewb4s44mfyryu

Seeing eye-to-eye? A comparison of object recognition performance in humans and deep convolutional neural networks under image manipulation [article]

Leonard E. van Dyck, Walter R. Gruber
2020 arXiv   pre-print
For a considerable time, deep convolutional neural networks (DCNNs) have reached human benchmark performance in object recognition.  ...  This study aims towards a behavioral comparison of visual core object recognition performance between humans and feedforward neural networks in a classification learning paradigm on an ImageNet data set  ...  Deep Neural Networks In order to investigate DCNN performance, the three neural networks AlexNet, GoogLeNet, and ResNet-50 were trained and tested entirely in MATLAB with Deep Learning Toolbox (Version  ... 
arXiv:2007.06294v2 fatcat:tsvnp7zv7ja6fasofm3hbqg7tq
« Previous Showing results 1 — 15 out of 6,657 results