996 Hits in 4.6 sec

Instance adaptive adversarial training: Improved accuracy tradeoffs in neural nets [article]

Yogesh Balaji, Tom Goldstein, Judy Hoffman
2019 arXiv   pre-print
Adversarial training is by far the most successful strategy for improving robustness of neural networks to adversarial attacks.  ...  We show that using our approach, test accuracy on unperturbed samples improve with a marginal drop in robustness.  ...  ACKNOWLEDGEMENTS Goldstein and Balaji were supported in part by the DARPA GARD program, DARPA QED for RML, DARPA Lifelong Learning Machines, the DARPA Young Faculty Award program, the AFOSR MURI program  ... 
arXiv:1910.08051v1 fatcat:s2enskjbwbazbi4azpfnak4zte

Calibrated Adversarial Training [article]

Tianjin Huang, Vlado Menkovski, Yulong Pei, Mykola Pechenizkiy
2021 arXiv   pre-print
Adversarial training is an approach of increasing the robustness of models to adversarial attacks by including adversarial examples in the training set.  ...  In this paper, we present the Calibrated Adversarial Training, a method that reduces the adverse effects of semantic perturbations in adversarial training.  ...  (a) Intersect (b) Intersect (c) Non-Intersect (d) Non-Intersect Improved accuracy tradeoffs in neural nets. arXiv preprint arXiv: bounds.  ... 
arXiv:2110.00623v2 fatcat:tvqzw5tgzffrle4l4etdjezxle

Improved Adversarial Training via Learned Optimizer [article]

Yuanhao Xiong, Cho-Jui Hsieh
2020 arXiv   pre-print
Then we leverage a learning-to-learn (L2L) framework to train an optimizer with recurrent neural networks, providing update directions and steps adaptively for the inner problem.  ...  By co-training optimizer's parameters and model's weights, the proposed framework consistently improves the model robustness over PGD-based adversarial training and TRADES.  ...  In the meanwhile, we train two surrogate models: one is Plain-Net with natural training and the other is PGD-Net with 10-step PGD-based adversarial training. Results are presented in Table 7 .  ... 
arXiv:2004.12227v1 fatcat:d6lp75ouk5e53flbvdlhhjrfc4

Incorporating human and learned domain knowledge into training deep neural networks: A differentiable dose volume histogram and adversarial inspired framework for generating Pareto optimal dose distributions in radiation therapy [article]

Dan Nguyen, Rafe McBeth, Azar Sadeghnejad Barkousaraie, Gyanendra Bohara, Chenyang Shen, Xun Jia, Steve Jiang
2019 arXiv   pre-print
The mean squared error (MSE) loss, dose volume histogram (DVH) loss, and adversarial (ADV) loss were used to train 4 instances of the neural network model: 1) MSE, 2) MSE+ADV, 3) MSE+DVH, and 4) MSE+DVH  ...  Expert human domain specific knowledge can be the largest driver in the performance improvement, and adversarial learning can be used to further capture nuanced features.  ...  Loss Functions In this study, 3 loss functions-mean squared error (MSE) loss, dose volume histogram (DVH) loss, and adversarial (ADV) loss-were used to train and compare 4 instances of the neural network  ... 
arXiv:1908.05874v2 fatcat:4ubnp3yca5g77asb5txirlfepm

AugMax: Adversarial Composition of Random Augmentations for Robust Training [article]

Haotao Wang, Chaowei Xiao, Jean Kossaifi, Zhiding Yu, Anima Anandkumar, Zhangyang Wang
2022 arXiv   pre-print
Data augmentation is a simple yet effective way to improve the robustness of deep neural networks (DNNs).  ...  For example, AugMix explores random compositions of a diverse set of augmentations to enhance broader coverage, while adversarial training generates adversarially hard samples to spot the weakness.  ...  [26] augment the training set with samples from a fictitious adversarial domain to improve domain adaptation performance. Xie et al.  ... 
arXiv:2110.13771v3 fatcat:e7ulbwviprhyzpoyt72m2tlzem

Data Quality Matters For Adversarial Training: An Empirical Study [article]

Chengyu Dong, Liyuan Liu, Jingbo Shang
2021 arXiv   pre-print
Multiple intriguing problems are hovering in adversarial training, including robust overfitting, robustness overestimation, and robustness-accuracy trade-off.  ...  We then design controlled experiments to investigate the interconnections between data quality and problems in adversarial training.  ...  Improving classification accuracy by identifying and removing instances that should be misclassified. The 2011 International Joint Conference on Neural Networks, pp. J. Chen and Quanquan Gu.  ... 
arXiv:2102.07437v3 fatcat:uwotwqcmtndqnaubjyvt5bof6i

Privacy and Fairness in Recommender Systems via Adversarial Training of User Representations [article]

Yehezkel S. Resheff, Yanai Elazar, Moni Shahar, Oren Sar Shalom
2018 arXiv   pre-print
Privacy risks of such systems have previously been studied mostly in the context of recovery of personal information in the form of usage records from the training data.  ...  In this paper we show that user vectors calculated by a common recommender system can be exploited in this way.  ...  We adapt the adversarial training framework in the context of privacy in recommender systems.  ... 
arXiv:1807.03521v3 fatcat:2myhg3taabgljino3rwjzzmk6q

Understanding Generalization in Adversarial Training via the Bias-Variance Decomposition [article]

Yaodong Yu, Zitong Yang, Edgar Dobriban, Jacob Steinhardt, Yi Ma
2021 arXiv   pre-print
This underscores the power of bias-variance decompositions in modern settings-by providing two measurements instead of one, they can rule out more explanations than test accuracy alone.  ...  Adversarially trained models exhibit a large generalization gap: they can interpolate the training set even for large perturbation radii, but at the cost of large test error on clean samples.  ...  Related work Robustness-accuracy tradeoff.  ... 
arXiv:2103.09947v2 fatcat:xa45kg3ykjgcje5qtmue6rblia

FasTrCaps: An Integrated Framework for Fast yet Accurate Training of Capsule Networks [article]

Alberto Marchisio, Beatrice Bussolino, Alessio Colucci, Muhammad Abdullah Hanif, Maurizio Martina, Guido Masera, Muhammad Shafique
2020 arXiv   pre-print
We demonstrate that one of the solutions generated by the FasTrCaps framework can achieve 58.6% reduction in the training time, while preserving the accuracy (even 0.12% accuracy improvement for the MNIST  ...  In this paper, we implement different optimizations in the training loop of the CapsNets, and investigate how these optimizations affect their training speed and the accuracy.  ...  learning rate, while providing a reduction of 79.31% in the training time. 2) A more extensive training with warm restarts leads to to an accuracy improvement of 0.07%. 3) The adaptive batch size shows  ... 
arXiv:1905.10142v2 fatcat:itirbilzcvh77nqg3k6g4zf2zq

Search Spaces for Neural Model Training [article]

Darko Stosic, Dusan Stosic
2021 arXiv   pre-print
While larger neural models are pushing the boundaries of what deep learning can do, often more weights are needed to train models rather than to run inference for tasks.  ...  training more effective.  ...  The fact accuracy improves with more training suggests neural models are often not trained to capacity using conventional schedules.  ... 
arXiv:2105.12920v1 fatcat:hkhqf3z3wjdpjf4ijnyukddt7m

A Broad Study of Pre-training for Domain Generalization and Adaptation [article]

Donghyun Kim, Kaihong Wang, Stan Sclaroff, Kate Saenko
2022 arXiv   pre-print
In this paper, we provide a broad study and in-depth analysis of pre-training for domain adaptation and generalization, namely: network architectures, size, pre-training loss, and datasets.  ...  We observe that simply using a state-of-the-art backbone outperforms existing state-of-the-art domain adaptation baselines and set new baselines on Office-Home and DomainNet improving by 10.7\% and 5.5  ...  In Sec. 4.2, while we find that adaptation methods still improve the accuracy with modern pre-training, the relative ranking of domain adaptation methods is not preserved.  ... 
arXiv:2203.11819v2 fatcat:7hdtwqiqd5ht3p52ab4stzphnm

In-N-Out: Pre-Training and Self-Training using Auxiliary Information for Out-of-Distribution Robustness [article]

Sang Michael Xie, Ananya Kumar, Robbie Jones, Fereshte Khani, Tengyu Ma, Percy Liang
2021 arXiv   pre-print
error but can hurt OOD error; but (ii) using auxiliary information as outputs of auxiliary pre-training tasks improves OOD error.  ...  To get the best of both worlds, we introduce In-N-Out, which first trains a model with auxiliary inputs and uses it to pseudolabel all the in-distribution inputs, then pre-trains a model on OOD auxiliary  ...  Raghunathan et al. (2020) analyze robust self-training (RST) (Carmon et al., 2019; Najafi et al., 2019; Uesato et al., 2019), which improves the tradeoff between standard and adversarially robust accuracy  ... 
arXiv:2012.04550v3 fatcat:va3qhyxnjzhnbgfozkt7blm72y

Band-limited Training and Inference for Convolutional Neural Networks [article]

Adam Dziedzic and John Paparrizos and Sanjay Krishnan and Aaron Elmore and Michael Franklin
2019 arXiv   pre-print
In particular, we found: (1) band-limited training can effectively control the resource usage (GPU and memory); (2) models trained with band-limited layers retain high prediction accuracy; and (3) requires  ...  The convolutional layers are core building blocks of neural network architectures. In general, a convolutional filter applies to the entire frequency spectrum of the input data.  ...  This research was supported in part by the Center for Unstoppable Computing (CERES) at University of Chicago, NSF CISE Expeditions Award CCF-1139158, generous support from Google and NVIDIA, and the NSF  ... 
arXiv:1911.09287v1 fatcat:kqowv5qxl5fvrkcpfcalmj2nmu

Certified Defenses: Why Tighter Relaxations May Hurt Training [article]

Nikola Jovanović, Mislav Balunović, Maximilian Baader, Martin Vechev
2021 arXiv   pre-print
In contrast, in this paper we study the underlying causes and show that tightness alone may not be the determining factor.  ...  Certified defenses based on convex relaxations are an established technique for training provably robust models.  ...  For instance, while tightening the relaxation via an adaptive lower bound in CROWN improved certification, it created the discontinuity issue, leading to inferior results in training.  ... 
arXiv:2102.06700v2 fatcat:f3rbasvxc5hdzjhbaolrbpmoea

Normalization Techniques in Training DNNs: Methodology, Analysis and Application [article]

Lei Huang, Jie Qin, Yi Zhou, Fan Zhu, Li Liu, Ling Shao
2020 arXiv   pre-print
Normalization techniques are essential for accelerating the training and improving the generalization of deep neural networks (DNNs), and have successfully been used in various applications.  ...  This paper reviews and comments on the past, present and future of normalization methods in the context of DNN training.  ...  Similar ideas have also been exploited in unsupervised adversarial domain adaptation in the context of semantic scene segmentation [232] and adversarial examples for improving image recognition [233  ... 
arXiv:2009.12836v1 fatcat:fei3jdfm2rajfdzqdmjghmmjsq
« Previous Showing results 1 — 15 out of 996 results