Filters








61,970 Hits in 6.1 sec

Depth Separations in Neural Networks: What is Actually Being Separated? [article]

Itay Safran, Ronen Eldan, Ohad Shamir
2021 arXiv   pre-print
Existing depth separation results for constant-depth networks essentially show that certain radial functions in ℝ^d, which can be easily approximated with depth 3 networks, cannot be approximated by depth  ...  2 networks, even up to constant accuracy, unless their size is exponential in d.  ...  Acknowledgements This research is supported in part by European Research Council (ERC) Grant 754705.  ... 
arXiv:1904.06984v3 fatcat:2yxq6ep7grgmfbu24m3mha2oye

Size and Depth Separation in Approximating Benign Functions with Neural Networks [article]

Gal Vardi, Daniel Reichman, Toniann Pitassi, Ohad Shamir
2021 arXiv   pre-print
It implies that beyond depth 4 there is a barrier to showing depth-separation for benign functions, even between networks of constant depth and networks of nonconstant depth.  ...  When studying the expressive power of neural networks, a main challenge is to understand how the size and depth of the network affect its ability to approximate real functions.  ...  This research is supported in part by European Research Council (ERC) grant 754705. References S. Arora and B. Barak. Computational complexity: a modern approach. Cambridge University Press, 2009.  ... 
arXiv:2102.00314v3 fatcat:v56bn7msrfe6ngltmxkhiqggr4

The Connection Between Approximation, Depth Separation and Learnability in Neural Networks [article]

Eran Malach, Gilad Yehudai, Shai Shalev-Shwartz, Ohad Shamir
2021 arXiv   pre-print
Specifically, we show that a necessary condition for a function to be learnable by gradient descent on deep neural networks is to be able to approximate the function, at least in a weak sense, with shallow  ...  neural networks.  ...  ACKNOWLEDGMENTS This research is supported by the European Research Council (TheoryDL project), and by European Research Council (ERC) grant 754705.  ... 
arXiv:2102.00434v2 fatcat:xzhrh4mi75fqpej7pe76d2z6iy

Neural Networks with Small Weights and Depth-Separation Barriers [article]

Gal Vardi, Ohad Shamir
2020 arXiv   pre-print
In studying the expressiveness of neural networks, an important question is whether there are functions which can only be approximated by sufficiently deep networks, assuming their size is bounded.  ...  constant-depth neural networks.  ...  Acknowledgements This research is supported in part by European Research Council (ERC) grant 754705.  ... 
arXiv:2006.00625v4 fatcat:hmfiaolo4rfpzljomlkw2sfyzy

Building separable approximations for quantum states via neural networks [article]

Antoine Girardin, Nicolas Brunner, Tamás Kriváchy
2022 arXiv   pre-print
Moreover, we show our method to be efficient in the multipartite case, considering different notions of separability.  ...  To tackle this task, we parametrize separable states with a neural network and train it to minimize the distance to a given target state, with respect to a differentiable distance, such as the trace distance  ...  This is precisely what we observe. We run the neural network independently for 11 values of q, and additionally for the exact separability boundary value.  ... 
arXiv:2112.08055v4 fatcat:65w4evwv3ngyzdglpmyzia3mia

Dissipative Hamiltonian Neural Networks: Learning Dissipative and Conservative Dynamics Separately [article]

Andrew Sosanya, Sam Greydanus
2022 arXiv   pre-print
Recent work has shown that neural networks can learn such symmetries directly from data using Hamiltonian Neural Networks (HNNs).  ...  But HNNs struggle when trained on datasets where energy is not conserved. In this paper, we ask whether it is possible to identify and decompose conservative and dissipative dynamics simultaneously.  ...  The authors would also like to thank Robyn Millan, Kristina Lynch, and Mathias Van Regemortel for their encouragement in this intellectual pursuit.  ... 
arXiv:2201.10085v2 fatcat:e4zhx2w5mncjticka7t2cphz54

Building separable approximations for quantum states via neural networks

Antoine Girardin, Nicolas Brunner, Tamás Kriváchy
2022 Physical Review Research  
Moreover, we show our method to be efficient in the multipartite case, considering different notions of separability.  ...  To tackle this task, we parametrize separable states with a neural network and train it to minimize the distance to a given target state with respect to a differentiable distance, such as the trace distance  ...  This is precisely what we observe. We run the neural network independently for 11 values of q and additionally for the exact separability boundary value.  ... 
doi:10.1103/physrevresearch.4.023238 fatcat:jd3cwpzvvrcbddsywooe4suk6a

Video Classification with Channel-Separated Convolutional Networks [article]

Du Tran and Heng Wang and Lorenzo Torresani and Matt Feiszli
2019 arXiv   pre-print
It is natural to ask: 1) if group convolution can help to alleviate the high computational cost of video classification networks; 2) what factors matter the most in 3D group convolutional networks; and  ...  These two empirical findings lead us to design an architecture -- Channel-Separated Convolutional Network (CSN) -- which is simple, efficient, yet accurate.  ...  We introduce the term "channel-separated" to highlight the importance of channel interaction; we also point out that the existing term "depth-separable" is only a good description when applied to tensors  ... 
arXiv:1904.02811v4 fatcat:dlj24sqqfvfynhrlq2ilqfur44

Network architecture underlying maximal separation of neuronal representations

Ron A. Jortner
2013 Frontiers in Neuroengineering  
sparse and selective codes and linking network architecture and neural coding.  ...  The antennallobe is a small network: ∼800 excitatory PNs which send their axons to the next relays in the system (forming the antennal-lobe's sole output), and ∼300 inhibitory interneurons (not shown in  ...  To see what happens when neural activity is added in, let us put some flesh on the dry skeleton, and proceed to explore the aggregate input to KCs ( k ) during network activitycorresponding to their sub-threshold  ... 
doi:10.3389/fneng.2012.00019 pmid:23316159 pmcid:PMC3539730 fatcat:h7fddjmwprew5lnyzv27sxljkq

An Efficient and Accurate Depth-Wise Separable Convolutional Neural Network for Cybersecurity Vulnerability Assessment Based on CAPTCHA Breaking

Stephen Dankwa, Lu Yang
2021 Electronics  
new training dataset, and then proposed an efficient and accurate Depth-wise Separable Convolutional Neural Network for breaking text-based CAPTCHAs.  ...  Most importantly, to the best of our knowledge, this is the first CAPTCHA breaking study to use the Depth-wise Separable Convolution layer to build an efficient CNN model to break text-based CAPTCHAs.  ...  And lastly, we thank all the Lab mates in 905Lab of School of Automation Engineering for their encouragements. Conflicts of Interest: The authors declare no conflict of interest.  ... 
doi:10.3390/electronics10040480 fatcat:l6aapljfbzcbtjykq4dwbjc2qq

Separating Figure from Ground with a Parallel Network

Paul K Kienker, Terrence J Sejnowski, Geoffrey E Hinton, Lee E Schumacher
1986 Perception  
Our network model is too simplified to serve as a model of human performance, but it does demonstrate that one global property of outlines can be computed through local interactions in a parallel network  ...  The rapidity with which we can discriminate the inside from the outside of a figure suggests that at least this step in the process may be performed in visual cortex by a large number of neurons in several  ...  The speed with which the decision can be made compared with the time scale of neural processing suggests that figureground separation is computed in parallel over the visual field.  ... 
doi:10.1068/p150197 pmid:3774489 fatcat:zat2cutujzfs5o2jwloibwdhee

Small-Scale Depthwise Separable Convolutional Neural Networks for Bacteria Classification

Duc-Tho Mai, Koichiro Ishibashi
2021 Electronics  
neural networks.  ...  The number of parameters in this architecture is seven times less than the smallest model listed in the literature.  ...  In convolutional neural networks, non-convex functions often need to be optimized.  ... 
doi:10.3390/electronics10233005 fatcat:pr3gaxnn3zfo3ajv5u3vrha3v4

The Separation Capacity of Random Neural Networks [article]

Sjoerd Dirksen, Martin Genzel, Laurent Jacques, Alexander Stollenwerk
2021 arXiv   pre-print
In the present article we enhance the theoretical understanding of random neural nets by addressing the following data separation problem: under what conditions can a random neural network make two classes  ...  to fully learned neural networks.  ...  Introduction Despite the unprecedented success of neural networks (NNs) in countless applications [LBH15; Sch15; GBC16], a rigorous understanding of their operating principles is still in its infancy.  ... 
arXiv:2108.00207v1 fatcat:fyicfnb6wna6xacz4fujnkbnfi

Semi-Blind Approaches for Source Separation and Independent component Analysis

Massoud Babaie-Zadeh, Christian Jutten
2006 The European Symposium on Artificial Neural Networks  
This paper is a survey of semi-blind source separation approaches.  ...  Although providing a generic framework for semi-blind source separation, Sparse Component Analysis and Bayesian ICA will just sketched in this paper, since two other survey papers develop in depth these  ...  In [40] , a geometrical approach (similar to what is presented in the previous section) is presented for separating discrete (n-valued) sources, in which the independence of the sources is not required  ... 
dblp:conf/esann/Babaie-ZadehJ06 fatcat:lbedh4fmvnabhbgtya2eewmimq

Detecting Deepfake-Forged Contents with Separable Convolutional Neural Network and Image Segmentation [article]

Chia-Mu Yu, Ching-Tang Chang, Yen-Wu Ti
2019 arXiv   pre-print
The proposed solution reaches detection efficiency by using the recently proposed separable convolutional neural network (CNN) and image segmentation.  ...  Deepfake is a variant of auto-encoders that use deep learning techniques to identify and exchange images of a person's face in a picture or film.  ...  In the proposed network architecture, the input is a 256 × 256 × 3 pixel image. The tensor is obtained by a convolution network with a length and width of 32, and a depth of 128.  ... 
arXiv:1912.12184v1 fatcat:hfrvho3i65gwjnporxosdr3bp4
« Previous Showing results 1 — 15 out of 61,970 results