570 Hits in 3.1 sec

Fast Rates for Online Prediction with Abstention [article]

Gergely Neu, Nikita Zhivotovskiy
2020 arXiv   pre-print
We exactly characterize the dependence on the abstention cost c and the number of experts N by providing matching upper and lower bounds of order log N/1-2c, which is to be contrasted with the best possible  ...  and the fast rates mentioned above, under some natural assumptions on the sequence of abstention costs.  ...  The protocol of online binary prediction with abstentions For each round t = 1, 2, . . . , T , repeat: 1.  ... 
arXiv:2001.10623v2 fatcat:rllvzrcdxrarxi7ng64ub4huyy

Active learning from noisy and abstention feedback

Songbai Yan, Kamalika Chaudhuri, Tara Javidi
2015 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton)  
We provide a model for this setting where the abstention rate and the noise rate increase as we get closer to the decision boundary of the ground truth hypothesis.  ...  We provide an algorithm and an analysis of the number of queries it makes to the labeling oracle; finally we provide matching lower bounds to demonstrate that our algorithm has near-optimal estimation  ...  PRELIMINARIES We consider active learning for binary classification.  ... 
doi:10.1109/allerton.2015.7447165 dblp:conf/allerton/YanCJ15 fatcat:igz7ybcnwjbmhavkjzomn7r2ey

On the use of ROC analysis for the optimization of abstaining classifiers

Tadeusz Pietraszek
2007 Machine Learning  
These classifiers are built based on optimization criteria in the following three models: cost-based, bounded-abstention and bounded-improvement.  ...  We propose a method to optimally build a specific type of abstaining binary classifiers using ROC analysis.  ...  In the bounded-abstention model, the boundary condition is the maximum abstention rate of the classifier, whereas in the bounded-improvement model, the boundary condition is the maximum misclassification  ... 
doi:10.1007/s10994-007-5013-y fatcat:dwtoywoea5efhmf3n5hvva5kru

Consistency in models for distributed learning under communication constraints

J.B. Predd, S.R. Kulkarni, H.V. Poor
2006 IEEE Transactions on Information Theory  
For several basic communication models in both the binary classification and regression frameworks, we question the existence of agent decision rules and fusion rules that result in a universally consistent  ...  The answers to this question present new issues to consider with regard to universal consistency.  ...  In Sections III, and IV, we study the models for binary classification in communication with and without abstention, respectively.  ... 
doi:10.1109/tit.2005.860420 fatcat:rcxuu5m5svfrtjfpunwkzfyzoi

Towards optimally abstaining from prediction with OOD test examples [article]

Adam Tauman Kalai, Varun Kanade
2021 arXiv   pre-print
Our work builds on a recent abstention algorithm of Goldwasser, Kalais, and Montasser (2020) for transductive binary classification.  ...  In particular, our transductive abstention algorithm takes labeled training examples and unlabeled test examples as input, and provides predictions with optimal prediction loss guarantees.  ...  This implies E[ 1 + 2 ] ≤ Õ( d/n), D Transductive bounds for binary classification with Q = P In this section, we bound the expected worst-case test error for binary classification, given training and  ... 
arXiv:2105.14119v2 fatcat:dj5lvmoqd5euznols6zzp5g4wa

MOBA: A multi-objective bounded-abstention model for two-class cost-sensitive problems [article]

Hongjiao Guan
2019 arXiv   pre-print
Specifically, the MOBA model minimizes the error rate of each class under class-dependent abstention constraints.  ...  To overcome such problems, a multi-objective bounded-abstention (MOBA) model is proposed to optimize essential metrics.  ...  The general classification rule with reject option in binary classification is shown in Eq. (1) and can be explained as follows.  ... 
arXiv:1905.07297v1 fatcat:khfqegggwnc55c742rmixlosmi

Structured Output Learning with Abstention: Application to Accurate Opinion Prediction [article]

Alexandre Garcia, Slim Essid, Chloé Clavel, Florence d'Alché-Buc
2018 arXiv   pre-print
Moreover, the abstention-aware representations can be used to competitively predict user-review ratings based on a sentence-level opinion predictor.  ...  To compare fully labeled training data with predictions potentially containing abstentions, we define a wide class of asymmetric abstention-aware losses.  ...  In Binary classification with abstention, we have Y = {0, 1} and the abstention-aware loss ∆ bin a is defined by : ∆ bin a (h(x), r(x), y) =      1 if y = h(x) and r(x) = 1 0 if y = h(x) and r(x)  ... 
arXiv:1803.08355v2 fatcat:yngxcievebbrvi2mph3esacy4m

Optimizing abstaining classifiers using ROC analysis

Tadeusz Pietraszek
2005 Proceedings of the 22nd international conference on Machine learning - ICML '05  
These classifiers are built based on optimization criteria in the following three models: cost-based, bounded-abstention and bounded-improvement.  ...  We propose a method to optimally build a specific type of abstaining binary classifiers using ROC analysis.  ...  , 2000) ), the bounded-abstention model, and the bounded-improvement model.  ... 
doi:10.1145/1102351.1102435 dblp:conf/icml/Pietraszek05 fatcat:ovgctrvtcvhq7p4zzrs54hxczy

A General Framework for Abstention Under Label Shift [article]

Amr M. Alexandari, Anshul Kundaje, Avanti Shrikumar
2022 arXiv   pre-print
In this work, we present a general framework for abstention that can be applied to optimize any metric of interest, that is adaptable to label shift at test time, and that works out-of-the-box with any  ...  Standard abstention methods tend to be focused on optimizing top-k accuracy, but in many applications, accuracy is not the metric of interest.  ...  binary classification tasks) [Corces et al., 2016] .  ... 
arXiv:1802.07024v5 fatcat:i7dx6yhyjnfojll3lg6lihnz3y

Fast classification rates without standard margin assumptions [article]

Olivier Bousquet, Nikita Zhivotovskiy
2020 arXiv   pre-print
Based on those results, we derive the necessary and sufficient conditions for classification (without a reject option) with fast rates in the agnostic setting achievable by improper learners.  ...  First, we consider classification with a reject option, namely Chow's reject option model, and show that by slightly lowering the impact of hard instances, a learning rate of order O(d/nlogn/d) is always  ...  Classification with a reject option (with abstentions).  ... 
arXiv:1910.12756v2 fatcat:e7cokll6mfeq3lwla4rbenlnma

Selective Ensembles for Consistent Predictions [article]

Emily Black and Klas Leino and Matt Fredrikson
2021 arXiv   pre-print
low abstention rates.  ...  On several benchmark datasets, selective ensembles reach zero inconsistently predicted points, with abstention rates as low 1.5%.  ...  ACKNOWLEDGMENTS This work was developed with the support of NSF grant CNS-1704845, NSF CNS-1943016, as well as by DARPA and the Air Force Research Laboratory under agreement number FA8750-15-2-0277.  ... 
arXiv:2111.08230v1 fatcat:atlkwbmicna6xaya27fnoksiui

Consistency in Models for Communication Constrained Distributed Learning [chapter]

J. B. Predd, S. R. Kulkarni, H. V. Poor
2004 Lecture Notes in Computer Science  
., the asymptotics of several agent decision rules and fusion rules are considered in both binary classification and regression frameworks.  ...  The agents are limited in their ability to communicate to a fusion center; the amount of information available for classification or regression is constrained.  ...  Distributed Classification with Abstention: Stone's Theorem In this section, we show that the universal consistency of distributed classification with abstention follows immediately from Stone's Theorem  ... 
doi:10.1007/978-3-540-27819-1_31 fatcat:qnzykkyb6fe57ha45alxj2qkla

The Utility of Abstaining in Binary Classification [article]

Akshay Balsubramani
2015 arXiv   pre-print
We explore the problem of binary classification in machine learning, with a twist - the classifier is allowed to abstain on any datum, professing ignorance about the true class label without committing  ...  We review efficient algorithms with provable guarantees for each of these areas.  ...  The Mistake Bound Model The Mistake Bound (MB) model [Lit88] for binary classification is a general framework for the study of online binary classification algorithms.  ... 
arXiv:1512.08133v1 fatcat:q7ekqcjdsnahnjmgpgpzdwqrba

PAC-Bayes with Minimax for Confidence-Rated Transduction [article]

Akshay Balsubramani, Yoav Freund
2015 arXiv   pre-print
We derive minimax optimal rules for confidence-rated prediction in this setting.  ...  We consider using an ensemble of binary classifiers for transductive prediction, when unlabeled test data are known in advance.  ...  Introduction Modern applications of binary classification have recently driven renewed theoretical interest in the problem of confidence-rated prediction [1, 2, 3] .  ... 
arXiv:1501.03838v1 fatcat:zk5o4sz2gfglbo3mpsobtbligy

Abstaining Classification When Error Costs are Unequal and Unknown [article]

Hongjiao Guan, Yingtao Zhang, H. D. Cheng, Xianglong Tang
2018 arXiv   pre-print
In this paper, we propose a bounded-abstention method with two constraints of reject rates (BA2), which performs abstaining classification when error costs are unequal and unknown.  ...  Meanwhile, BA2 achieves controllable reject rates of the positive and negative classes.  ...  In this paper, we propose a ROC-based abstaining classification method, bounded-abstention with two constraints of reject rates (BA2), to overcome the limitations of using posterior probabilities and cost  ... 
arXiv:1806.03445v2 fatcat:ohy3mpd7sngzhjd3q6dthgu3hm
« Previous Showing results 1 — 15 out of 570 results