A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Filters
No True State-of-the-Art? OOD Detection Methods are Inconsistent across Datasets
[article]
2021
arXiv
pre-print
Preserved Fulltext
Prior literature has proposed various methods (e.g., MSP (Hendrycks & Gimpel, 2017), ODIN (Liang et al., 2018), Mahalanobis (Lee et al., 2018)), claiming they are state-of-the-art by showing they outperform ...
In this work, we show that none of these methods are inherently better at OOD detection than others on a standardized set of 16 (ID, OOD) pairs. ...
No method does
consistently better than any other method across the 16 pairs (none
of the numbers are 0 or 16). ...
arXiv:2109.05554v1
fatcat:6uwo3vaj7ngjjjt3ilf2xeoqbq
DeepGaze IIE: Calibrated prediction in and out-of-domain for state-of-the-art saliency modeling
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Other Versions
This new model, "DeepGaze IIE", yields a significant leap in benchmark performance in and out-of-domain with a 15 percent point improvement over DeepGaze II to 93% on MIT1003, marking a new state of the ...
art on the MIT/Tuebingen Saliency Benchmark in all available metrics (AUC: 88.3%, sAUC: 79.4%, CC: 82.4%). ...
Our pairwise combination of models is already enough to beat the state of the art, while our final combination of four models with three instances each leads to an even higher leap on the state of the ...
arXiv:2105.12441v3
fatcat:srjm65isxnbt3imgri2vvcfdjm
Are all outliers alike? On Understanding the Diversity of Outliers for Detecting OODs
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Deep neural networks (DNNs) are known to produce incorrect predictions with very high confidence on out-of-distribution (OOD) inputs. ...
Our results include experiments on CIFAR10, SVHN and MNIST as in-distribution data and Imagenet, LSUN, SVHN (for CIFAR10), CIFAR10 (for SVHN), KMNIST, and F-MNIST as OOD data across different DNN architectures ...
A.2.2 COMPARISON WITH THE STATE-OF-THE-ART OOD DETECTION METHODS IN SUPERVISED SETTINGS ON PRE-TRAINED CLASSIFIERS We compare our results with the state-of-the-art methods in supervised settings, as reported ...
arXiv:2103.12628v1
fatcat:ej4w6j2q4bae7mnm27aycc2mui
A Simple Framework for Robust Out-of-Distribution Detection
2022
IEEE Access
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
The existing state-of-the-art methods of OOD detection tackle this issue by utilizing the internal feature of the classification network. ...
Our experiments demonstrate the superiority of our simple method under various OOD detection scenarios. ...
TOWARDS BETTER OOD BENCHMARKS This section points out that existing state-of-the-art methods [20] , [28] in out-of-distribution (OOD) detection do not generalize well on detecting hard OOD datasets ...
doi:10.1109/access.2022.3153723
fatcat:e6bodhr7fnblpejcyr42zoht2m
Norm-Scaling for Out-of-Distribution Detection
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Out-of-Distribution (OoD) inputs are examples that do not belong to the true underlying distribution of the dataset. ...
We show that norm-scaling, when used with maximum softmax probability detector, achieves 9.78% improvement in AUROC, 5.99% improvement in AUPR and 33.19% reduction in FPR95 metrics over previous state-of-the-art ...
On average, norm-scaling on MSP achieves 9.78% improvement in AU-ROC, 5.99% improvement in AUPR and 33.19% reduction in FPR95 over previous state-of-the-art methods. ...
arXiv:2205.03493v1
fatcat:3l2mbtiqtfgsnoofwgabzdpxka
Confidence-based Out-of-Distribution Detection: A Comparative Study and Analysis
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
In this work, we assess the capability of various state-of-the-art approaches for confidence-based OOD detection through a comparative study and in-depth analysis. ...
First, we leverage a computer vision benchmark to reproduce and compare multiple OOD detection methods. ...
Conclusion This work presented an analysis of various state-of-the-art methods for confidencebased OOD detection on a computer vision and a medical imaging task. ...
arXiv:2107.02568v1
fatcat:26z53wikwjezvfkznxezke2g2i
Memory-Efficient Semi-Supervised Continual Learning: The World is its Own Replay Buffer
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Note that this fulltext copy is not of the "primary" version of this work. The version it corresponds to is:
2021 pre-print arXiv:2101.09536v1 fatcat:hxwn2z3cpjbrxlnu77lwnvhw5i
Our approach, DistillMatch, increases performance over the state-of-the-art by no less than 8.7% average task accuracy and up to 54.5% average task accuracy in SSCL CIFAR-100 experiments. ...
We show that a strategy built on pseudo-labeling, consistency regularization, Out-of-Distribution (OoD) detection, and knowledge distillation reduces forgetting in this setting. ...
We show that state-of-the-art continual learning methods perform inconsistently in the SSCL setting (i.e. no baseline method is "best" across all settings). 2. ...
arXiv:2101.09536v2
fatcat:inyfzbclgfbw7mrej4cxh5q5ze
Estimating Predictive Uncertainty Under Program Data Distribution Shift
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
We also propose a large-scale benchmark of existing state-of-the-art predictive uncertainty on programming tasks and investigate their effectiveness under data distribution shift. ...
on program dataset. ...
However, to our knowledge all the existing state-of-the-art work evaluates the effectiveness of their uncertainty methods on CV and NLP tasks. ...
arXiv:2107.10989v1
fatcat:vxjuenmwhngefioz7ja23o4gc4
Scaling Out-of-Distribution Detection for Real-World Settings
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Note that this fulltext copy is not of the "primary" version of this work. The version it corresponds to is:
2022 pre-print arXiv:1911.11132v3 fatcat:obsy3ko4bfdu7jkexi2b4pjogeOther Versions
We conduct extensive experiments in these more realistic settings for out-of-distribution detection and find that a surprisingly simple detector based on the maximum logit outperforms prior methods in ...
To test ImageNet multiclass anomaly detectors, we introduce the Species dataset containing over 700,000 images and over a thousand anomalous species. ...
Assuming single-model evaluation and no access to other anomalies or test-time adaptation, the MSP attains state-of-the-art anomaly detection performance in small-scale settings. ...
arXiv:1911.11132v4
fatcat:6d235jsui5cjbciufvbxn65vcy
Detecting Out-of-distribution Samples via Variational Auto-encoder with Reliable Uncertainty Estimation
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Experiments on various datasets demonstrate that compared to the standard VAEs, our model is superior in uncertainty estimation for the OOD data and is robust in anomaly detection tasks. ...
Variational autoencoders (VAEs) are influential generative models with rich representation capabilities from the deep neural network architecture and Bayesian method. ...
Through a number of experiments on the challenging OOD cases, we demonstrate that INCPVAE can learn the true characterization of OOD inputs, and achieves state-of-the-art (SOTA) performance in OOD detection ...
arXiv:2007.08128v3
fatcat:ovh6tdtwwbgo7a7qbtv4z3sggq
Confidence from Invariance to Image Transformations
[article]
2018
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
In addition, we apply our technique to novelty detection scenarios, where we also demonstrate state of the art results. ...
In experiments with multiple data sets (STL-10,CIFAR-100,ImageNet) and classifiers, we demonstrate new state of the art for the error detection task. ...
In addition, we apply our technique to novelty detection scenarios, where we also demonstrate state of the art results. ...
arXiv:1804.00657v1
fatcat:o3xdnjq4ljazjebxmq22hlilwm
Data-SUITE: Data-centric identification of in-distribution incongruous examples
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Other Versions
We also illustrate how these identified regions can provide insights into datasets and highlight their limitations. ...
B.1.3 Deep Ensembles (ENSEMBLE) Deep Ensembles [Lakshminarayanan et al., 2017] is widely regarded as the state-of-the-art non-Bayesian uncertainty estimation method. ...
For each of the detection methods, we compute the overlap between the predicted OOD/Outlier instances and the uncertain and inconsistent instances as identified by Data-SUITE. ...
arXiv:2202.08836v2
fatcat:vh7tzoy6jzcprc643osxo6qtce
Relevant-features based Auxiliary Cells for Energy Efficient Detection of Natural Errors
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Deep neural networks have demonstrated state-of-the-art performance on many classification tasks. However, they have no inherent capability to recognize when their predictions are wrong. ...
We demonstrate the effectiveness of our technique on various image classification datasets such as CIFAR-10, CIFAR-100 and Tiny-ImageNet. ...
We demonstrate the effectiveness of our methodology to detect natural errors on state-of-the-art networks, such as VGG [18] and ResNet [5] for image classification tasks on CIFAR [1] and Tiny-ImageNet ...
arXiv:2002.11052v2
fatcat:ukmefarv55cwdpwducpbazdti4
Detecting Out-of-Distribution Examples with In-distribution Examples and Gram Matrices
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
performs better than or equal to state-of-the-art OOD detection methods (including those that do assume access to OOD examples). ...
The method is applicable across a variety of architectures and vision datasets and, for the important and surprisingly hard task of detecting far-from-distribution out-of-distribution examples, it generally ...
We have proposed and reported on a relatively simple OOD detection method based on pairwise feature correlations that gives new state of the art detection results without requiring access to anything other ...
arXiv:1912.12510v2
fatcat:7y34yyxenbfjxcblxlap2h7hze
Improving Self-supervised Learning with Automated Unsupervised Outlier Arbitration
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
We empirically show UOTA's advantage over the state-of-the-art self-supervised paradigms with evident margin, which well justifies the existence of the OOD sample issue embedded in the existing approaches ...
Our work reveals a structured shortcoming of the existing mainstream self-supervised learning methods. ...
Acknowledgments and Disclosure of Funding Funding in direct support of this work: the National Key R&D Program of China under Grant No. 2020AAA0108600. ...
arXiv:2112.08132v1
fatcat:swknm76jwfe43cofmr2zknyusu
« Previous
Showing results 1 — 15 out of 138 results