A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Filters
Instance adaptive adversarial training: Improved accuracy tradeoffs in neural nets
[article]
2019
arXiv
pre-print
Preserved Fulltext
Adversarial training is by far the most successful strategy for improving robustness of neural networks to adversarial attacks. ...
We show that using our approach, test accuracy on unperturbed samples improve with a marginal drop in robustness. ...
ACKNOWLEDGEMENTS Goldstein and Balaji were supported in part by the DARPA GARD program, DARPA QED for RML, DARPA Lifelong Learning Machines, the DARPA Young Faculty Award program, the AFOSR MURI program ...
arXiv:1910.08051v1
fatcat:s2enskjbwbazbi4azpfnak4zte
Calibrated Adversarial Training
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Adversarial training is an approach of increasing the robustness of models to adversarial attacks by including adversarial examples in the training set. ...
In this paper, we present the Calibrated Adversarial Training, a method that reduces the adverse effects of semantic perturbations in adversarial training. ...
(a) Intersect (b) Intersect (c) Non-Intersect (d) Non-Intersect Improved accuracy tradeoffs in neural nets. arXiv preprint arXiv: bounds. ...
arXiv:2110.00623v2
fatcat:tvqzw5tgzffrle4l4etdjezxle
Improved Adversarial Training via Learned Optimizer
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Then we leverage a learning-to-learn (L2L) framework to train an optimizer with recurrent neural networks, providing update directions and steps adaptively for the inner problem. ...
By co-training optimizer's parameters and model's weights, the proposed framework consistently improves the model robustness over PGD-based adversarial training and TRADES. ...
In the meanwhile, we train two surrogate models: one is Plain-Net with natural training and the other is PGD-Net with 10-step PGD-based adversarial training. Results are presented in Table 7 . ...
arXiv:2004.12227v1
fatcat:d6lp75ouk5e53flbvdlhhjrfc4
Incorporating human and learned domain knowledge into training deep neural networks: A differentiable dose volume histogram and adversarial inspired framework for generating Pareto optimal dose distributions in radiation therapy
[article]
2019
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
The mean squared error (MSE) loss, dose volume histogram (DVH) loss, and adversarial (ADV) loss were used to train 4 instances of the neural network model: 1) MSE, 2) MSE+ADV, 3) MSE+DVH, and 4) MSE+DVH ...
Expert human domain specific knowledge can be the largest driver in the performance improvement, and adversarial learning can be used to further capture nuanced features. ...
Loss Functions In this study, 3 loss functions-mean squared error (MSE) loss, dose volume histogram (DVH) loss, and adversarial (ADV) loss-were used to train and compare 4 instances of the neural network ...
arXiv:1908.05874v2
fatcat:4ubnp3yca5g77asb5txirlfepm
AugMax: Adversarial Composition of Random Augmentations for Robust Training
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Note that this fulltext copy is not of the "primary" version of this work. The version it corresponds to is:
2021 pre-print arXiv:2110.13771v2 fatcat:k6wk56bu3fghxj3xpxtmkdn2yeOther Versions
Data augmentation is a simple yet effective way to improve the robustness of deep neural networks (DNNs). ...
For example, AugMix explores random compositions of a diverse set of augmentations to enhance broader coverage, while adversarial training generates adversarially hard samples to spot the weakness. ...
[26] augment the training set with samples from a fictitious adversarial domain to improve domain adaptation performance. Xie et al. ...
arXiv:2110.13771v3
fatcat:e7ulbwviprhyzpoyt72m2tlzem
Data Quality Matters For Adversarial Training: An Empirical Study
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Multiple intriguing problems are hovering in adversarial training, including robust overfitting, robustness overestimation, and robustness-accuracy trade-off. ...
We then design controlled experiments to investigate the interconnections between data quality and problems in adversarial training. ...
Improving classification accuracy by identifying and removing instances that should be misclassified. The 2011 International Joint Conference on Neural Networks, pp. J. Chen and Quanquan Gu. ...
arXiv:2102.07437v3
fatcat:uwotwqcmtndqnaubjyvt5bof6i
Privacy and Fairness in Recommender Systems via Adversarial Training of User Representations
[article]
2018
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Privacy risks of such systems have previously been studied mostly in the context of recovery of personal information in the form of usage records from the training data. ...
In this paper we show that user vectors calculated by a common recommender system can be exploited in this way. ...
We adapt the adversarial training framework in the context of privacy in recommender systems. ...
arXiv:1807.03521v3
fatcat:2myhg3taabgljino3rwjzzmk6q
Understanding Generalization in Adversarial Training via the Bias-Variance Decomposition
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Other Versions
This underscores the power of bias-variance decompositions in modern settings-by providing two measurements instead of one, they can rule out more explanations than test accuracy alone. ...
Adversarially trained models exhibit a large generalization gap: they can interpolate the training set even for large perturbation radii, but at the cost of large test error on clean samples. ...
Related work Robustness-accuracy tradeoff. ...
arXiv:2103.09947v2
fatcat:xa45kg3ykjgcje5qtmue6rblia
FasTrCaps: An Integrated Framework for Fast yet Accurate Training of Capsule Networks
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
We demonstrate that one of the solutions generated by the FasTrCaps framework can achieve 58.6% reduction in the training time, while preserving the accuracy (even 0.12% accuracy improvement for the MNIST ...
In this paper, we implement different optimizations in the training loop of the CapsNets, and investigate how these optimizations affect their training speed and the accuracy. ...
learning rate, while providing a reduction of 79.31% in the training time. 2) A more extensive training with warm restarts leads to to an accuracy improvement of 0.07%. 3) The adaptive batch size shows ...
arXiv:1905.10142v2
fatcat:itirbilzcvh77nqg3k6g4zf2zq
Search Spaces for Neural Model Training
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
While larger neural models are pushing the boundaries of what deep learning can do, often more weights are needed to train models rather than to run inference for tasks. ...
training more effective. ...
The fact accuracy improves with more training suggests neural models are often not trained to capacity using conventional schedules. ...
arXiv:2105.12920v1
fatcat:hkhqf3z3wjdpjf4ijnyukddt7m
A Broad Study of Pre-training for Domain Generalization and Adaptation
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Note that this fulltext copy is not of the "primary" version of this work. The version it corresponds to is:
2022 pre-print arXiv:2203.11819v1 fatcat:iuz2hrdn4narvn6yumpmsh6rxe
In this paper, we provide a broad study and in-depth analysis of pre-training for domain adaptation and generalization, namely: network architectures, size, pre-training loss, and datasets. ...
We observe that simply using a state-of-the-art backbone outperforms existing state-of-the-art domain adaptation baselines and set new baselines on Office-Home and DomainNet improving by 10.7\% and 5.5 ...
In Sec. 4.2, while we find that adaptation methods still improve the accuracy with modern pre-training, the relative ranking of domain adaptation methods is not preserved. ...
arXiv:2203.11819v2
fatcat:7hdtwqiqd5ht3p52ab4stzphnm
In-N-Out: Pre-Training and Self-Training using Auxiliary Information for Out-of-Distribution Robustness
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Other Versions
error but can hurt OOD error; but (ii) using auxiliary information as outputs of auxiliary pre-training tasks improves OOD error. ...
To get the best of both worlds, we introduce In-N-Out, which first trains a model with auxiliary inputs and uses it to pseudolabel all the in-distribution inputs, then pre-trains a model on OOD auxiliary ...
Raghunathan et al. (2020) analyze robust self-training (RST) (Carmon et al., 2019; Najafi et al., 2019; Uesato et al., 2019), which improves the tradeoff between standard and adversarially robust accuracy ...
arXiv:2012.04550v3
fatcat:va3qhyxnjzhnbgfozkt7blm72y
Band-limited Training and Inference for Convolutional Neural Networks
[article]
2019
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
In particular, we found: (1) band-limited training can effectively control the resource usage (GPU and memory); (2) models trained with band-limited layers retain high prediction accuracy; and (3) requires ...
The convolutional layers are core building blocks of neural network architectures. In general, a convolutional filter applies to the entire frequency spectrum of the input data. ...
This research was supported in part by the Center for Unstoppable Computing (CERES) at University of Chicago, NSF CISE Expeditions Award CCF-1139158, generous support from Google and NVIDIA, and the NSF ...
arXiv:1911.09287v1
fatcat:kqowv5qxl5fvrkcpfcalmj2nmu
Certified Defenses: Why Tighter Relaxations May Hurt Training
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Other Versions
In contrast, in this paper we study the underlying causes and show that tightness alone may not be the determining factor. ...
Certified defenses based on convex relaxations are an established technique for training provably robust models. ...
For instance, while tightening the relaxation via an adaptive lower bound in CROWN improved certification, it created the discontinuity issue, leading to inferior results in training. ...
arXiv:2102.06700v2
fatcat:f3rbasvxc5hdzjhbaolrbpmoea
Normalization Techniques in Training DNNs: Methodology, Analysis and Application
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Normalization techniques are essential for accelerating the training and improving the generalization of deep neural networks (DNNs), and have successfully been used in various applications. ...
This paper reviews and comments on the past, present and future of normalization methods in the context of DNN training. ...
Similar ideas have also been exploited in unsupervised adversarial domain adaptation in the context of semantic scene segmentation [232] and adversarial examples for improving image recognition [233 ...
arXiv:2009.12836v1
fatcat:fei3jdfm2rajfdzqdmjghmmjsq
« Previous
Showing results 1 — 15 out of 996 results