A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Filters
Do Neural Networks Compress Manifolds Optimally?
[article]
2022
arXiv
pre-print
Preserved Fulltext
Artificial Neural-Network-based (ANN-based) lossy compressors have recently obtained striking results on several sources. ...
In contrast, we determine the optimal entropy-distortion tradeoffs for two low-dimensional manifolds with circular structure and show that state-of-the-art ANN-based compressors fail to optimally compress ...
The analysis and synthesis transforms are fully-connected feedforward neural networks with 2 hidden layers containing 100 neurons each. ...
arXiv:2205.08518v1
fatcat:rh7mel342fdyjixq2fs5od2kpi
Optimal Stable Nonlinear Approximation
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
A measure of optimal performance, called stable manifold widths, for approximating a model class K in a Banach space X by stable manifold methods is introduced. ...
The effects of requiring stability in the settings of deep learning and compressed sensing are discussed. ...
This is discussed in §6 for compressed sensing and neural network approximation. ...
arXiv:2009.09907v1
fatcat:wbkxyqw6cffgnit2ybulrjfpfa
Extrapolative Bayesian Optimization with Gaussian Process and Neural Network Ensemble Surrogate Models
2021
Advanced Intelligent Systems
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Herein, various surrogate models for BO, including GPs and neural network ensembles (NNEs), are investigated. ...
Two materials datasets of different complexity with different properties are used, to compare the performance of GP and NNE-the first is the compressive strength of concrete (8 inputs and 1 target), and ...
[30] In yet another implementation, Abolhasani et al. utilized randomly generated neural network architectures to produce an ensemble of neural networks as the surrogate model. ...
doi:10.1002/aisy.202100101
fatcat:tnmlfhgsyven5lgjywk7oexpdq
Adversarial Attacks on Face Detectors using Neural Net based Constrained Optimization
[article]
2018
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
In this paper, we propose a novel strategy to craft adversarial examples by solving a constrained optimization problem using an adversarial generator network. ...
In a different experiment, also on 300-W, we demonstrate the robustness of our attack to a JPEG compression based defense typical JPEG compression level of 75% reduces the effectiveness of our attack from ...
One theory for a JPEG based defense are that adversarial examples lie off the data manifold under which neural networks are so successful and by using JPEG compression the adversarial examples are projected ...
arXiv:1805.12302v1
fatcat:a4sy3gswbjb23kc6h55qrj5qkq
Overcoming Catastrophic Forgetting via Direction-Constrained Optimization
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Other Versions
This paper studies a new design of the optimization algorithm for training deep learning models with a fixed architecture of the classification network in a continual learning framework. ...
Furthermore, in order to control the memory growth as the number of tasks increases, we propose a memory-efficient version of our algorithm called compressed DCO (DCO-COMP) that allocates a memory of fixed ...
., network parameters) that correspond to good performance of the network on all encountered tasks determine a common manifold of plausible solutions for all these optimization problems. ...
arXiv:2011.12581v2
fatcat:vvdhj72d7vggljvoyzhdh3w5wy
ZoPE: A Fast Optimizer for ReLU Networks with Low-Dimensional Inputs
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Other Versions
We demonstrate the versatility of the optimizer in analyzing networks by projecting onto the range of a generative adversarial network and visualizing the differences between a compressed and uncompressed ...
Using ZoPE, we observe a 25× speedup on property 1 of the ACAS Xu neural network verification benchmark compared to several state-of-the-art verifiers, and an 85× speedup on a set of linear optimization ...
Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of DARPA, any NASA entity, or the National Science ...
arXiv:2106.05325v2
fatcat:psvcsb42xjbw5at4nm5fp6kwfm
Optimizing Grouped Convolutions on Edge Devices
2020
2020 IEEE 31st International Conference on Application-specific Systems, Architectures and Processors (ASAP)
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
When deploying a deep neural network on constrained hardware, it is possible to replace the network's standard convolutions with grouped convolutions. ...
However, current implementations of grouped convolutions in modern deep learning frameworks are far from performing optimally in terms of speed. ...
The opinions expressed and arguments employed herein do not necessarily reflect the official views of these funding bodies. ...
doi:10.1109/asap49362.2020.00039
dblp:conf/asap/GibsonCTCOS20
fatcat:gt2mb2n3sbgj7hbddbbsqkn5je
Optimizing Grouped Convolutions on Edge Devices
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
When deploying a deep neural network on constrained hardware, it is possible to replace the network's standard convolutions with grouped convolutions. ...
However, current implementations of grouped convolutions in modern deep learning frameworks are far from performing optimally in terms of speed. ...
• We evaluate the performance of GSPC using different network models on the CPU of several edge devices. • We compare GSPC against implementations of grouped convolutions present in widely used deep learning ...
arXiv:2006.09791v1
fatcat:prpnrh3y2fevnclcz32flkcjmq
Instance-Optimal Compressed Sensing via Posterior Sampling
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
We show for Gaussian measurements and any prior distribution on the signal, that the posterior sampling estimator achieves near-optimal recovery guarantees. ...
We characterize the measurement complexity of compressed sensing of signals drawn from a known prior distribution, even when the support of the prior is the entire space (rather than, say, sparse vectors ...
Learned d-
amp: Principled neural network based compressive image
recovery. In Advances in Neural Information Processing
Systems, pp. 1772-1783, 2017.
Mosser, L., Dubrule, O., and Blunt, M. J. ...
arXiv:2106.11438v1
fatcat:n6tqdezb7zgrnal36iqznydy64
An Optimal Control Approach to Deep Learning and Applications to Discrete-Weight Neural Networks
[article]
2018
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
This allows one to characterize necessary conditions for optimality and develop training algorithms that do not rely on gradients with respect to the trainable parameters. ...
The developed methods are applied to train, in a rather principled way, neural networks with weights that are constrained to take values in a discrete set. ...
manifolds must contain an optimal solution, if one exists. ...
arXiv:1803.01299v2
fatcat:5rhxonq5sjfcngkj7y3wozs53q
Adversarial Concurrent Training: Optimizing Robustness and Accuracy Trade-off of Deep Neural Networks
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
However, there seems to be an inherent trade-off between optimizing the model for accuracy and robustness. ...
We demonstrate the effectiveness of the proposed approach across different datasets and network architectures. ...
Model complexity The magnitudes of the weights of neural networks can provide an estimate of the model's complexity. ...
arXiv:2008.07015v2
fatcat:avwvknudcvcfhmfek7fw7ntyqu
Optimal Approximation with Sparsely Connected Deep Neural Networks
[article]
2018
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Finally, we present numerical experiments demonstrating that the standard stochastic gradient descent algorithm generates deep neural networks providing close-to-optimal approximation rates. ...
Specifically, all function classes that are optimally approximated by a general class of representation systems---so-called affine systems---can be approximated by deep neural networks with minimal connectivity ...
are also optimally representable by neural networks. ...
arXiv:1705.01714v4
fatcat:xyht6tpa2rehhdozqhth2e6wui
Model compression as constrained optimization, with application to neural nets. Part V: combining compressions
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Experimentally with deep neural nets, we observe that 1) we can find significantly better models in the error-compression space, indicating that different compression types have complementary benefits, ...
We formulate this generally as a problem of optimizing the loss but where the weights are constrained to equal an additive combination of separately compressed parts; and we give an algorithm to learn ...
A basic issue is the representation ability of the compression: given an optimal point in model space (the weight parameters for a neural net), which manifold or subset of this space can be compressed ...
arXiv:2107.04380v1
fatcat:yxfpaug3uzam7lhlhaaxebpomm
Reverse engineering learned optimizers reveals known and novel mechanisms
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Learned optimizers are algorithms that can themselves be trained to solve optimization problems. ...
Our results help elucidate the previously murky understanding of how learned optimizers work, and establish tools for interpreting future learned optimizers. ...
a fully connected neural network on the two moons dataset, and (d) training a convolutional neural network on the MNIST dataset. ...
arXiv:2011.02159v2
fatcat:n7ciopm3nvfm5ome3ngv3za5qm
Latent Space Arc Therapy Optimization
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Traditionally, heuristics such as fluence-map-optimization-informed segment initialization use locally optimal solutions to begin the search of the full arc therapy plan space from a reasonable starting ...
Volumetric modulated arc therapy planning is a challenging problem in high-dimensional, non-convex optimization. ...
The canonical example of unsupervised learning with neural networks is deep autoencoders (AE) [18] . ...
arXiv:2106.05846v1
fatcat:hgdepv3q2jhz3fux7cyirx3v5q
« Previous
Showing results 1 — 15 out of 6,801 results