1,085 Hits in 4.2 sec

Misclassification Risk and Uncertainty Quantification in Deep Classifiers

Murat Sensoy, Maryam Saleki, Simon Julier, Reyhan Aydogan, John Reid
2021 2021 IEEE Winter Conference on Applications of Computer Vision (WACV)  
In this paper, we propose risk-calibrated evidential deep classifiers to reduce the costs associated with classification errors. We use two main approaches.  ...  We demonstrate that our approach allows to (i) incorporate misclassification cost while training deep classifiers, (ii) accurately quantify the uncertainty of classification predictions, and (iii) simultaneously  ...  Among those, evidential deep learning is the state-of-the-art and a practical approach for uncertainty quantification in deep classifiers [18] .  ... 
doi:10.1109/wacv48630.2021.00253 fatcat:uzi2qsdlsjhi5k6jyreet5vi64

Uncertainty-aware Personal Assistant for Making Personalized Privacy Decisions [article]

Gonul Ayci, Murat Sensoy, Arzucan Özgür, Pınar Yolum
2022 arXiv   pre-print
Accordingly, this paper proposes a personal assistant that uses evidential deep learning to classify content based on its privacy label.  ...  An important characteristic of the personal assistant is that it can model its uncertainty in its decisions explicitly, determine that it does not know the answer, and delegate from making a recommendation  ...  There may be different ways of incorporating the user's risk of misclassification into the training of evidential classifiers.  ... 
arXiv:2205.06544v4 fatcat:r6y3ihxecbe4zawltg2bl4rgda

URSABench: Comprehensive Benchmarking of Approximate Bayesian Inference Methods for Deep Neural Networks [article]

Meet P. Vadera, Adam D. Cobb, Brian Jalaian, Benjamin M. Marlin
2020 arXiv   pre-print
to quantify uncertainty and their robustness.  ...  In this paper, we describe initial work on the development ofURSABench(the Uncertainty, Robustness, Scalability, and Accu-racy Benchmark), an open-source suite of bench-marking tools for comprehensive  ...  The views and conclusions contained in this document are those  ... 
arXiv:2007.04466v1 fatcat:6hics7m6onhj7g6gztgydildsq

Probabilistic Neighbourhood Component Analysis: Sample Efficient Uncertainty Estimation in Deep Learning [article]

Ankur Mallick, Chaitanya Dwivedi, Bhavya Kailkhura, Gauri Joshi, T. Yong-Jin Han
2020 arXiv   pre-print
Our approach enables deep kNN classifier to accurately quantify underlying uncertainties in its prediction.  ...  Several uncertainty-aware models, such as Bayesian Neural Network (BNNs) and Deep Ensembles have been proposed in the literature for quantifying predictive uncertainty.  ...  By mapping data into distributions in a latent space before performing classification, we enable a deep kNN classifier to accurately quantify underlying uncertainties in its prediction.  ... 
arXiv:2007.10800v1 fatcat:nownqtyilvd27dcgr2jguq26am

Failure Prediction by Confidence Estimation of Uncertainty-Aware Dirichlet Networks [article]

Theodoros Tsiligkaridis
2020 arXiv   pre-print
In this paper, it is first shown that uncertainty-aware deep Dirichlet neural networks provide an improved separation between the confidence of correct and incorrect predictions in the true class probability  ...  Reliably assessing model confidence in deep learning and predicting errors likely to be made are key elements in providing safety for model deployment, in particular for applications with dire consequences  ...  Deep Dirichlet networks have been shown to outperform BNNs in uncertainty quantification for out-ofdistribution and adversarial queries [30, 31, 19] .  ... 
arXiv:2010.09865v1 fatcat:a5trhfwjrbb3nnhmijqri2fwsi

Leveraging Uncertainty in Deep Learning for Pancreatic Adenocarcinoma Grading [article]

Biraja Ghoshal, Bhargab Ghoshal, Allan Tucker
2022 arXiv   pre-print
In digital pathology, AI-based cancer grading must be extremely accurate in prediction and uncertainty quantification to improve reliability and explainability and are essential for gaining clinicians  ...  Specifically, it is useful in setting the acceptance threshold using a metric that weighs classification accuracy-reject trade-off and misclassification cost controlled by hyperparameters and can be employed  ...  We have also shown how to leverage estimated uncertainty in prediction as rejection threshold in classifying images by user-defined hyperparameters for a given cost of misclassification and rejection cost  ... 
arXiv:2206.08787v1 fatcat:otj24w77arh5dpt2lv5qpcjuw4

Deep Bayesian U-Nets for Efficient, Robust and Reliable Post-Disaster Damage Localization [article]

Xiao Liang, Seyed Omid Sajedi
2020 arXiv   pre-print
This paper is dedicated to developing deep Bayesian U-Nets where the uncertainty of predictions is a second output of the model, which is made possible through Monte Carlo dropout sampling in test time  ...  The existing models in the literature only provide a final prediction output, while the risks of utilizing such models for safety-critical assessments should not be ignored.  ...  Uncertainty Quantification Up to this point, we have shown that Bayesian U-Net is a robust classifier that can be used to localize damage.  ... 
arXiv:2009.11460v1 fatcat:rdygb7pkvbazrco45vugfi27gy

Benchmarking Uncertainty Estimation Methods for Deep Learning With Safety-Related Metrics

Maximilian Henne, Adrian Schwaiger, Karsten Roscher, Gereon Weiss
2020 AAAI Conference on Artificial Intelligence  
In this paper we attempt to fill this gap by evaluating and comparing several state-of-the-art methods for estimating uncertainty for image classifcation with respect to safety-related requirements and  ...  In particular, Deep Ensembles and Learned Confidence show high potential to significantly reduce the remaining error with only moderate performance penalties.  ...  the framework of "BAYERN DIGI-TAL II" and within the Intel Collaborative Research Institute Safe Automated Vehicles.  ... 
dblp:conf/aaai/HenneSRW20 fatcat:js566fszdnaqziac6isydpc2im

Estimation and Applications of Quantiles in Deep Binary Classification [article]

Anuj Tambwekar, Anirudh Maiya, Soma Dhavala, Snehanshu Saha
2021 arXiv   pre-print
The conditional quantiles provide a robust alternative to classical conditional means, and also allow uncertainty quantification of the predictions, while making very few distributional assumptions.  ...  We quantify the uncertainty of the class probabilities in terms of prediction intervals, and develop individualized confidence scores that can be used to decide whether a prediction is reliable or not  ...  Probal Choudhury, ISI Kolkata for suggestions and critical insights which helped the manuscript immensely. Anuj and Anirudh would like to thank Inumella Sricharan for his help, and Dr.  ... 
arXiv:2102.06575v1 fatcat:ebu57wvajne6hgux2y4tsipowm

An Uncertainty-Aware Deep Learning Framework for Defect Detection in Casting Products [article]

Maryam Habibpour, Hassan Gharoun, AmirReza Tajally, Afshar Shamsi, Hamzeh Asgharnezhad, Abbas Khosravi, Saeid Nahavandi
2021 arXiv   pre-print
Secondly, to achieve a reliable classification and to measure epistemic uncertainty, we employ an uncertainty quantification (UQ) technique (ensemble of MLP models) using features extracted from four pre-trained  ...  Howbeit, CNNs with frequentist inference require a massive amount of data to train on and still fall short in reporting beneficial estimates of their predictive uncertainty.  ...  In addition to uncertainty estimation, both correct classification and misclassification prediction groups are delineated in each plot.  ... 
arXiv:2107.11643v1 fatcat:v6oyqc2x4jcoxikp5v4fl5jevy

DAUX: a Density-based Approach for Uncertainty eXplanations [article]

Hao Sun, Boris van Breugel, Jonathan Crabbe, Nabeel Seedat, Mihaela van der Schaar
2022 arXiv   pre-print
: out-of-distribution (OOD) examples, boundary (Bnd) examples, and examples in regions of high in-distribution misclassification (IDM).  ...  Uncertainty quantification (UQ) is essential for creating trustworthy machine learning models.  ...  We use UCI's Covtype and Digits datasets (Dua & Graff, 2017) and focus on linear models (i.e. g = Id), since linear models are widely applied in settings where uncertainty, risk and misclassification  ... 
arXiv:2207.05161v1 fatcat:sfhry3uxqrb7tpbd7jfw43gzh4

Failure Detection in Medical Image Classification: A Reality Check and Benchmarking Testbed [article]

Melanie Bernhardt, Fabio De Sousa Ribeiro, Ben Glocker
2022 arXiv   pre-print
modalities, in multiclass and binary classification settings.  ...  This paper provides a reality check, establishing the performance of in-domain misclassification detection methods, benchmarking 9 confidence scores on 6 medical imaging datasets with different imaging  ...  to capture multi-modal solutions. • Deterministic Uncertainty Quantification (DUQ): Van Amersfoort et al. ( 2020 ) estimate predictive confidence based on distances between points and class centroids  ... 
arXiv:2205.14094v1 fatcat:ihyreztg65f4tgcyld4dhmsu4q

SCRIB: Set-Classifier with Class-Specific Risk Bounds for Blackbox Models

Zhen Lin, Lucas Glass, M. Brandon Westover, Cao Xiao, Jimeng Sun
Despite deep learning (DL) success in classification problems, DL classifiers do not provide a sound mechanism to decide when to refrain from predicting.  ...  We introduce Set-classifier with class-specific RIsk Bounds (SCRIB) to tackle this problem, assigning multiple labels to each example.  ...  Acknowledgments This work was supported by IQVIA, NSF award SCH-2014438, PPoSS 2028839, IIS-1838042, NIH award R01 1R01NS107291-01 and OSF Healthcare.  ... 
doi:10.1609/aaai.v36i7.20714 fatcat:5tf4l56qyfclnd4nq2bqgxcsia

Multidimensional Uncertainty-Aware Evidential Neural Networks [article]

Yibo Hu, Yuzhe Ou, Xujiang Zhao, Jin-Hee Cho, Feng Chen
2021 arXiv   pre-print
However, NNs have not considered inherent uncertainty in data associated with the class probabilities where misclassification under uncertainty may easily introduce high risk in decision making in real-world  ...  contexts (e.g., misclassification of objects in roads leads to serious accidents).  ...  Acknowledgments This work is supported by NSF awards IIS-1815696 and IIS-1750911.  ... 
arXiv:2012.13676v2 fatcat:pgw5dhooxvdkdeyzcs5dai6ehi

Uncertainty-Aware Deep Classifiers Using Generative Models

Murat Sensoy, Lance Kaplan, Federico Cerutti, Maryam Saleki
In this work we develop a novel neural network model that is able to express both aleatoric and epistemic uncertainty to distinguish decision boundary and out-of-distribution regions of the feature space  ...  Through extensive analysis, we demonstrate that the proposed approach provides better estimates of uncertainty for in- and out-of-distribution samples, and adversarial examples on well-known data sets  ...  The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the U.S.  ... 
doi:10.1609/aaai.v34i04.6015 fatcat:v5lveat4e5fxte3w4pvymkpmzi
« Previous Showing results 1 — 15 out of 1,085 results