986 Hits in 10.1 sec

Vapnik-Chervonenkis dimension of neural networks with binary weights

Stephan Mertens, Andreas Engel
1997 Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics  
We investigate the VC-dimension of the perceptron and simple two-layer networks like the committee- and the parity-machine with weights restricted to values ±1.  ...  For binary inputs, the VC-dimension is determined by atypical pattern sets, i.e. it cannot be found by replica analysis or numerical Monte Carlo sampling.  ...  For Nϭ7, the eighth column and row have been deleted, and for Nϭ9, a column with alternating Ϯ1's has been adjoined.  ... 
doi:10.1103/physreve.55.4478 fatcat:oi5f4wgtj5exxla6o23hqo6qr4

How Tight Are the Vapnik-Chervonenkis Bounds?

David Cohn, Gerald Tesauro
1992 Neural Computation  
For the N-input, single-layer networks, the VC-dimension of the net- work is N + 1.  ...  The lower bound on the dimension is just the number of weights in the network, and this is often used as a rough estimate of the actual VC-dimension.  ... 
doi:10.1162/neco.1992.4.2.249 fatcat:kbi7e2vtozggvngw5vlplykqze

VC Theoretical Explanation of Double Descent [article]

Eng Hock Lee, Vladimir Cherkassky
2022 arXiv   pre-print
In addition, we discuss several possible reasons for misinterpretation of VC-theoretical results in the machine learning community.  ...  We illustrate an application of analytic VC-bounds for modeling double descent for classification problems, using empirical results for several learning methods, such as SVM, Least Squares, and Multilayer  ...  Acknowledgments and Disclosure of Funding The authors would like to thank Dr. Vladimir Vapnik for providing valuable suggestions and insights on application of VC-bounds for modeling double descent.  ... 
arXiv:2205.15549v1 fatcat:ttlwvqh5lzd5hagxruokhhgpom

Learning from minimum entropy queries in a large committee machine

Peter Sollich
1996 Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics  
The connection between entropy and generalization error in multi-layer networks is discussed, and a computationally cheap algorithm for constructing queries is suggested and analysed.  ...  The generalization error decreases exponentially with the number of training examples, providing a significant improvement over the algebraic decay for random examples.  ...  We focus on one of the simplest multi-layer neural networks, namely, the tree-committee machine (TCM). A TCM is a two-layer network with N input units, K hidden units and one output unit.  ... 
doi:10.1103/physreve.53.r2060 pmid:9964607 fatcat:74ca5w2ir5fodbztmcu4kod32i

Sample Complexity Bounds for Recurrent Neural Networks with Application to Combinatorial Graph Problems [article]

Nil-Jana Akpinar, Bernhard Kratzwald, Stefan Feuerriegel
2019 arXiv   pre-print
While such derivations have been made earlier for feed-forward and convolutional neural networks, our work presents the first such attempt for recurrent neural networks.  ...  We further derive comparable results for multi-layer RNNs.  ...  With the aim of building on previous proof techniques for binary function classes, we relate the pseudo-dimension of a neural network to the VC-dimension of an extension of the network.  ... 
arXiv:1901.10289v2 fatcat:4dcgzxbgffcddmca6oaxwdvpxm

A theoretical framework for deep transfer learning

Tomer Galanti, Lior Wolf, Tamir Hazan
2016 Information and Inference A Journal of the IMA  
In the adversary model, we show that for binary classification, conventional PAClearning is equivalent to the new notion of PAC-transfer and to transfer generalization of the VC-dimension.  ...  In the randomized model, we provide PAC-Bayesian and VC-style generalization bounds to transfer learning, including bounds specifically derived for Deep Learning.  ...  This research was partly supported by a Grant from the GIF, the German-Israeli Foundation for Scientific Research and Development.  ... 
doi:10.1093/imaiai/iaw008 fatcat:dardamm4fndr5mu3qzz544deue

The Simultaneous Prediction of Soil Properties and Vegetation Coverage from Vis-NIR Hyperspectral Data with a One-Dimensional Convolutional Neural Network: A Laboratory Simulation Study

Fangfang Zhang, Changkun Wang, Kai Pan, Zhiying Guo, Jie Liu, Aiai Xu, Haiyi Ma, Xianzhang Pan
2022 Remote Sensing  
A model of a one-dimensional convolutional neural network (1DCNN) was established to simultaneously estimate soil properties (organic matter, soil moisture, clay, and sand) and vegetation coverage based  ...  Compared with the partial least squares regression (PLSR), the prediction accuracy of 1DCNN improved 42.20%, 45.82%, 43.32%, and 36.46% in terms of the root-mean-squared error (RMSE) for predicting soil  ...  Visualization of 32 filters weights for the first convolutional layer by the one-dimensional convolutional neural network (1DCNN) model for simultaneous prediction of soil properties and vegetation coverage  ... 
doi:10.3390/rs14020397 fatcat:5xbvuku3w5ajdfvuo2bik2kfyi

Learning Functions: When Is Deep Better Than Shallow [article]

Hrushikesh Mhaskar, Qianli Liao, Tomaso Poggio
2016 arXiv   pre-print
accuracy as shallow networks but with exponentially lower number of training parameters as well as VC-dimension.  ...  While the universal approximation property holds both for hierarchical and shallow networks, we prove that deep (hierarchical) networks can approximate the class of compositional functions with the same  ...  The VC-dimension of the shallow network with N units is bounded by (d + 2)N 2 ; the VC-dimension of the binary tree network with n(d − 1) units is bounded by 4n 2 (d − 1) 2 .  ... 
arXiv:1603.00988v4 fatcat:o5w4pcmyhfdkfmzbyq5ly37dgu

Using deep learning to investigate the neuroimaging correlates of psychiatric and neurological disorders: Methods and applications

Sandra Vieira, Walter H.L. Pinaya, Andrea Mechelli
2017 Neuroscience and Biobehavioral Reviews  
The results of these studies indicate that DL could be a powerful tool in the current search for biomarkers of psychiatric and neurologic disease.  ...  We conclude our review by discussing the main promises and challenges of using DL to elucidate brain-based disorders, as well as possible directions for future research.  ...  The layers between the input and output layers are called hidden layers, with the number of hidden layers representing the depth of the network.  ... 
doi:10.1016/j.neubiorev.2017.01.002 pmid:28087243 fatcat:orw3gi6scbhm3ct3r3y5lgojpa

Machine and quantum learning for diamond-based quantum applications [article]

Dylan G. Stone, Carlo Bradac
2022 arXiv   pre-print
We summarize some of the most prominent machine and quantum learning approaches that have been conducive to the presented advances and discuss their potential for proposed and future quantum applications  ...  In this work, we discuss and analyze the role machine and quantum learning are playing in the development of diamond-based quantum technologies.  ...  Acknowledgements The Natural Sciences and Engineering Research Council of Canada (via RGPIN-2021-03059, DGECR-2021-00234, USRA-574869-2022) and the Canada Foundation for Innovation (via John R.  ... 
arXiv:2208.00256v1 fatcat:jtrfkhmzvfcq3eij4knyvsthjy

Recent Advances in Neural Networks Structural Risk Minimization Based on Multiobjective Complexity Control Algorithms [chapter]

D.A.G. Vieira, J.A. Vasconcelos, R.R. Saldanh
2010 Machine Learning  
) −ỹ i ) 2 . (38) In Bartlett (1998) it was shown that the fat-shattering dimension, which is a generalization of the VC dimension, can be limited by limiting the weights of a given network.  ...  Following these considerations the SRM principle considers the minimization of two factors: the training error and the VC dimension.  ... 
doi:10.5772/9151 fatcat:2jmbadx52nemlisbrtcmfgosta

Congestion Control and Traffic Management in ATM Networks: Recent Advances and A Survey [article]

R. Jain
1998 arXiv   pre-print
In particular, selection criteria for selection between rate-based and credit-based approach and the key points of the debate between the two approaches are presented.  ...  Congestion control mechanisms for ATM networks as selected by the ATM Forum traffic management group are described. Reasons behind these selections are explained.  ...  David Hughes of StrataCom, and Rohit Goyal and Ram Viswanathan of OSU for useful feedback on an earlier version of this paper.  ... 
arXiv:cs/9809085v1 fatcat:7x4sqkazcfgyxoeoue77ppgyv4

Rademacher Complexity for Adversarially Robust Generalization [article]

Dong Yin and Kannan Ramchandran and Peter Bartlett
2020 arXiv   pre-print
We further consider a surrogate adversarial loss for one-hidden layer ReLU network and prove margin bounds for this setting.  ...  has an unavoidable dimension dependence, unless the weight vector has bounded ℓ_1 norm.  ...  The authors would like to thank Justin Gilmer for helpful discussion.  ... 
arXiv:1810.11914v4 fatcat:j64ipvkbsff2jpqfs7b5b7fetq

Capacity and Trainability in Recurrent Neural Networks [article]

Jasmine Collins, Jascha Sohl-Dickstein, David Sussillo
2017 arXiv   pre-print
We show experimentally that all common RNN architectures achieve nearly the same per-task and per-unit capacity bounds with careful training, for a variety of tasks and stacking depths.  ...  Two potential bottlenecks on the expressiveness of recurrent neural networks (RNNs) are their ability to store information about the task in their parameters, and to store information about the input history  ...  ACKNOWLEDGEMENTS We would like to thank Geoffrey Irving, Alex Alemi, Quoc Le, Navdeep Jaitly, and Taco Cohen for helpful feedback.  ... 
arXiv:1611.09913v3 fatcat:far6nus7trgcldgmmb2y3jdr7i

Detecting Memorization in ReLU Networks [article]

Edo Collins, Siavash Arjomand Bigdeli, Sabine Süsstrunk
2018 arXiv   pre-print
We propose a new notion of 'non-linearity' of a network layer with respect to an input batch that is based on its proximity to a linear system, which is reflected in the non-negative rank of the activation  ...  Furthermore, by applying our approach layer-by-layer, we find that the mechanism for memorization consists of distinct phases.  ...  Classical results in statistical learning consider properties of learning models such as the VC-dimension (Vapnik, 1998) and Rademacher complexity (Bartlett & Mendelson, 2002) .  ... 
arXiv:1810.03372v1 fatcat:3oq5kzomqbeq7pdkvd4xdpnsqm
« Previous Showing results 1 — 15 out of 986 results