Filters








663,349 Hits in 4.2 sec

Are Accuracy and Robustness Correlated

Andras Rozsa, Manuel Gunther, Terrance E. Boult
2016 2016 15th IEEE International Conference on Machine Learning and Applications (ICMLA)  
We find that adversarial examples are mostly transferable across similar network topologies, and we demonstrate that better machine learning models are less vulnerable to adversarial examples.  ...  We compare the adversarial example generation techniques with respect to the quality of the produced images, and measure the robustness of the tested machine learning models to adversarial examples.  ...  The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of the  ... 
doi:10.1109/icmla.2016.0045 dblp:conf/icmla/RozsaGB16 fatcat:5uipk3eixbez5bydk4qobk7usa

Estimating the Robustness of Classification Models by the Structure of the Learned Feature-Space [article]

Kalun Ho, Franz-Josef Pfreundt, Janis Keuper, Margret Keuper
2021 arXiv   pre-print
We introduce robustness indicators which are obtained via unsupervised clustering of latent representations from a trained classifier and show very high correlations to the model performance on corrupted  ...  While new benchmarks, like ImageNet-C, have been introduced to measure robustness properties, we argue that fixed testsets are only able to capture a small portion of possible data variations and are thus  ...  Furthermore, the columns ACC and P are showing the correlation between the model robustness and the clustering accuracy and purity, respectively.  ... 
arXiv:2106.12303v2 fatcat:l7kpjljeezexhdx7tdote4klzq

An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Models [article]

Lifu Tu, Garima Lalwani, Spandana Gella, He He
2020 arXiv   pre-print
Recent work has shown that pre-trained language models such as BERT improve robustness to spurious correlations in the dataset.  ...  When such minority examples are scarce, pre-trained models perform as poorly as models trained from scratch.  ...  Acknowledgments We would like to thank the Lex and Comprehend groups at Amazon Web Services AI for helpful discussions, and the reviewers for their insightful comments.  ... 
arXiv:2007.06778v3 fatcat:rwyfjjsx4fhojpjwdfdpsihanu

Ensemble Defense with Data Diversity: Weak Correlation Implies Strong Robustness [article]

Renjue Li, Hanwei Zhang, Pengfei Yang, Cheng-Chao Huang, Aimin Zhou, Bai Xue, Lijun Zhang
2021 arXiv   pre-print
From the theoretical perspective of DNN robustness, we argue that under the assumption of high quality of the filters, the weaker the correlations of the sensitivity of the filters are, the more robust  ...  Our ensemble models are more robust than those constructed by previous defense methods like adversarial training, and even competitive with the classical ensemble of adversarial trained DNNs under adversarial  ...  The minimum correlated ensemble model also has higher vote-based adversarial accuracy than the Gaussian ensemble model and the maximum correlated ensemble model.  ... 
arXiv:2106.02867v1 fatcat:wlc5cihqenezbmgsmq6ywnei3a

Robustness May Be at Odds with Accuracy [article]

Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, Aleksander Madry
2019 arXiv   pre-print
We demonstrate that this trade-off between the standard accuracy of a model and its robustness to adversarial perturbations provably exists in a fairly simple and natural setting.  ...  Specifically, training robust models may not only be more resource-consuming, but also lead to a reduction of standard accuracy.  ...  Acknowledgements Shibani Santurkar was supported by the National Science Foundation (NSF) under grants IIS-1447786, IIS-1607189, and CCF-1563880, and the Intel Corporation.  ... 
arXiv:1805.12152v5 fatcat:oy4xwgaclng7th3w2worocg6za

An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Models

Lifu Tu, Garima Lalwani, Spandana Gella, He He
2020 Transactions of the Association for Computational Linguistics  
Recent work has shown that pre-trained language models such as BERT improve robustness to spurious correlations in the dataset.  ...  When such minority examples are scarce, pre-trained models perform as poorly as models trained from scratch.  ...  Acknowledgments We would like to thank the Lex and Comprehend groups at Amazon Web Services AI for helpful 630 discussions, and the reviewers for their insightful comments.  ... 
doi:10.1162/tacl_a_00335 fatcat:5fvyazbdofawxb7beslhhppxme

Robust Phase Correlation based Motion Estimation and Its Applications

H. Yan, J.-G. Liu
2008 Procedings of the British Machine Vision Conference 2008  
A phase fringe filter and a highly robust estimator QMDPE are used to improve the fitting accuracy of the phase difference plane in Fourier domain.  ...  With the robust phase correlation and the compound phase correlation combined algorithm, the optical flow estimation and stereo matching achieved remarkable accuracy, especially around the areas with the  ...  technique and CPC method, are able to achieve remarkable accuracy in most synthetic and real images from different spectral bands.  ... 
doi:10.5244/c.22.104 dblp:conf/bmvc/YanL08 fatcat:fgz3vanenrewtlts44dbdiatre

Assaying Out-Of-Distribution Generalization in Transfer Learning [article]

Florian Wenzel, Andrea Dittadi, Peter Vincent Gehler, Carl-Johann Simon-Gabriel, Max Horn, Dominik Zietlow, David Kernert, Chris Russell, Thomas Brox, Bernt Schiele, Bernhard Schölkopf, Francesco Locatello
2022 arXiv   pre-print
Our findings confirm that in- and out-of-distribution accuracies tend to increase jointly, but show that their relation is largely dataset-dependent, and in general more nuanced and more complex than posited  ...  In this paper, we take a unified view of previous work, highlighting message discrepancies that we address empirically, and providing recommendations on how to measure the robustness of a model and how  ...  Florian and Andrea conducted the correlation analysis of robustness metrics. Carl-Johann and Chris conducted the factor analysis.  ... 
arXiv:2207.09239v1 fatcat:bwvxyg6yqvcyjdowkqcmoofr4a

Are Vision Transformers Robust to Spurious Correlations? [article]

Soumya Suvra Ghosal, Yifei Ming, Yixuan Li
2022 arXiv   pre-print
Our study reveals that when pre-trained on a sufficiently large dataset, ViT models are more robust to spurious correlations than CNNs.  ...  Further, we perform extensive ablations and experiments to understand the role of the self-attention mechanism in providing robustness under spuriously correlated environments.  ...  Both ViT-B/16 and ViT-S/16 attain better worst-group accuracy as compared to BiT models. All models are pre-trained on ImageNet-21k. Results (mean and std) are estimated over 3 runs for each setting.  ... 
arXiv:2203.09125v1 fatcat:krwiormjtrdctae3qfdzhqsv5a

Measuring Ensemble Diversity and Its Effects on Model Robustness

Lena Heidemann, Adrian Schwaiger, Karsten Roscher
2021 International Joint Conference on Artificial Intelligence  
And are diversity metrics suitable for selecting members to form a more robust ensemble?  ...  Deep ensembles have been shown to perform well on a variety of tasks in terms of accuracy, uncertainty estimation, and further robustness metrics.  ...  Acknowledgments This work was funded by the Bavarian Ministry for Economic Affairs, Regional Development and Energy as part of a project to support the thematic development of the Institute for Cognitive  ... 
dblp:conf/ijcai/HeidemannSR21 fatcat:62xcciseuvh7jep37raxiuzmjy

Convolutional Shallow Features for Performance Improvement of Histogram of Oriented Gradients in Visual Object Tracking

Suryo Adhi Wibowo, Hansoo Lee, Eun Kyeong Kim, Sungshin Kim
2017 Mathematical Problems in Engineering  
The results are then provided based on their accuracy-robustness (AR) rank.  ...  Furthermore, through a comparison with several state-of-the-art tracking algorithms, the proposed method was shown to achieve the highest rank in terms of accuracy and a third rank for robustness.  ...  Further, the accuracy and robustness ranks of the DFT tracker are also the same as those of the proposed method.  ... 
doi:10.1155/2017/6329864 fatcat:vkfopqldgvdldbrgqwzg2cscf4

Detecting correlation between server resources for system management

Stefania Tosi, Sara Casolari, Michele Colajanni
2014 Journal of computer and system sciences (Print)  
We compare the proposed solution and existing algorithms in terms of accuracy and robustness for several synthetic and real settings characterized by low and high variability, linear and non-linear correlation  ...  Unfortunately, the high variability characterizing most time series related to system resources affects the accuracy and robustness of existing correlation solutions.  ...  However, when relationships between time series are hidden by highly variable perturbations, the accuracy and robustness of existing correlation models are limited.  ... 
doi:10.1016/j.jcss.2014.01.002 fatcat:f322qabpsbe5xiahe4bglzgdvm

To What Degree is the Accuracy of a Bankruptcy Prediction Model Affected by the Environment? The Case of the Baltic States and the Czech Republic

Michal Karas, Mária Režňáková
2014 Procedia - Social and Behavioral Sciences  
An alternative solution to the problem of the limited transferability (e.g. robustness) of a model may be to identify the environmental factors that affect the model's prediction accuracy and incorporate  ...  This paper presents research of the prediction accuracy of a bankruptcy model in four countries and the correlation found between the development of selected macroeconomic indicators in these countries  ...  The correlations established at a five-percent significance level are highlighted in bold face.  ... 
doi:10.1016/j.sbspro.2014.11.241 fatcat:ggwjgj4ecve5pk2twelzi2bwve

Robust sub-pixel disparity estimation and its refinement around depth discontinuity and featureless areas

Hongshi Yan, Jian-Guo Liu
2010 2010 IEEE International Geoscience and Remote Sensing Symposium  
With the robust phase correlation disparity estimation and its refinement scheme, we are able to greatly improve the accuracy of phase correlation based disparity estimation.  ...  With this refinement scheme, we are able to greatly improve the accuracy of phase correlation based disparity estimation for DTM generation stereo image pairs with versatile baseline settings (from narrow  ... 
doi:10.1109/igarss.2010.5652314 dblp:conf/igarss/YanL10 fatcat:jw4lxu44y5gndb6ehzt7w3dl2y

Practical Assessment of Generalization Performance Robustness for Deep Networks via Contrastive Examples [article]

Xuanyu Wu, Xuhong Li, Haoyi Xiong, Xiao Zhang, Siyu Huang, Dejing Dou
2021 arXiv   pre-print
Our experiment results confirm that (1) behaviors of deep models on contrastive examples are strongly correlated to what on the testing set, and (2) ContRE is a robust measure of generalization performance  ...  Specifically, ContRE follows the assumption in contrastive learning that robust DNN models with good generalization performance are capable of extracting a consistent set of features and making consistent  ...  On the other hand, training accuracy is also related to the accuracy on contrastive examples because they are directly 5 The correlation between accuracy of training examples after random flip and original  ... 
arXiv:2106.10653v1 fatcat:tvxkdvoohvew7bo2urgxe3vqtu
« Previous Showing results 1 — 15 out of 663,349 results