11 Hits in 1.9 sec

Moment Multicalibration for Uncertainty Estimation [article]

Christopher Jung, Changhwa Lee, Mallesh M. Pai, Aaron Roth, Rakesh Vohra
2020 arXiv   pre-print
moment multicalibration has been obtained.  ...  We show how to achieve the notion of "multicalibration" from H\'ebert-Johnson et al. [2018] not just for means, but also for variances and other higher moments.  ...  Acknowledgements We are thankful for helpful early conversations with Sampath Kannan.  ... 
arXiv:2008.08037v1 fatcat:terbl5wpgfb3nhmbzdte2xawoq

Low-Degree Multicalibration [article]

Parikshit Gopalan, Michael P. Kim, Mihir Singhal, Shengjia Zhao
2022 arXiv   pre-print
Importantly, we show that low-degree multicalibration can be significantly more efficient than full multicalibration.  ...  In the multi-class setting, the sample complexity to achieve low-degree multicalibration improves exponentially (in the number of classes) over full multicalibration.  ...  The authors thank Gal Yona for useful feedback on an earlier draft of this manuscript and Omer Reingold and Udi Wieder for helpful discussions.  ... 
arXiv:2203.01255v2 fatcat:rcm7mpbhlrgiriplffxxm3yiwa

Beyond Bernoulli: Generating Random Outcomes that cannot be Distinguished from Nature

Cynthia Dwork, Michael P. Kim, Omer Reingold, Guy N. Rothblum, Gal Yona
2022 International Conference on Algorithmic Learning Theory  
(COLT 2021) on Moment Multicalibration.  ...  We find an equivalence between Outcome Indistinguishability and Multicalibration that is more subtle than in the binary case and sheds light on the techniques employed by Jung et al. to obtain Moment Multicalibration  ...  MPK is supported by the Miller Institute for Basic Research in Science.  ... 
dblp:conf/alt/DworkKRRY22 fatcat:zqg2ga3v2bdxvpl7itwkmjq6na

Prediction uncertainty validation for computational chemists [article]

Pascal Pernot
2022 arXiv   pre-print
Validation of prediction uncertainty (PU) is becoming an essential task for modern computational chemistry.  ...  However, its application is not limited to ML and it can serve as a principled framework for any PU validation.  ...  my analysis of the ZHE2022 dataset, and Matthew Evans for pointing out the Uncertainty Toolbox.  ... 
arXiv:2204.13477v7 fatcat:66i3fz3kdrckbalbs7v6dygzkq

Practical Adversarial Multivalid Conformal Prediction [article]

Osbert Bastani, Varun Gupta, Christopher Jung, Georgy Noarov, Ramya Ramalingam, Aaron Roth
2022 arXiv   pre-print
We call our algorithm MVP, short for MultiValid Prediction. We give both theory and an extensive set of empirical evaluations.  ...  We give a simple, generic conformal prediction method for sequential prediction that achieves target empirical coverage guarantees against adversarially chosen data.  ...  We thank Stephen Bates for helpful comments on an early version of this paper.  ... 
arXiv:2206.01067v1 fatcat:smbedyu2nfb6hok3z2qd6flvva

Online Multivalid Learning: Means, Moments, and Prediction Intervals

Varun Gupta, Christopher Jung, Georgy Noarov, Mallesh M. Pai, Aaron Roth, Mark Braverman
The second is variance and higher moment prediction, which corresponds to an online algorithm satisfying the notion of mean-conditioned moment multicalibration from [Jung et al., 2021].  ...  This means that the resulting estimates correctly predict various statistics of the labels y not just marginally - as averaged over the sequence of examples - but also conditionally on x ∈ G for any G  ...  We also thank Ashish Rastogi for discussions about uncertainty estimation in practice.  ... 
doi:10.4230/lipics.itcs.2022.82 fatcat:5bl6x5yx35bmbglgpgitdx3j54

About traceability of the biochemical measurement results – some case studies on the metrological and practical aspects

Steluta Duta, J.-R. Filtz, B. Larquier, P. Claudel, J.-O. Favreau
2013 16th International Congress of Metrology   unpublished
In laboratories medicine, the metrological traceability and the related metrological concept measurement uncertainty are not always possible to be established.  ...  In this paper some traceability case studies for biochemical constituents are presented in opposite with some case studies for hematology measurements performed in medical laboratories.  ...  It should be also mentioned that information regarding the uncertainty associated to the multicalibrators´s assigned values are missing, the measurement uncertainty components are not fully identified  ... 
doi:10.1051/metrology/201305007 fatcat:obntfbxq3jdhrhthh7l77mgh6y

Laboratory management, accreditation in laboratory medicine

2015 Clinical Chemistry and Laboratory Medicine  
For periods 1, 2 and 3 the average total TAT was 61, 59 and 53 minutes, respectively.  ...  The average TAT was 26, 19, 12 and 57 minutes for the preanalytical, analytical, postanalytical and total phases, respectively.  ...  Data included: estimation of %u rw from intralab imprecision (within-run and day to day), calculation of %u bias , combined uncertainty %u=(%u 2 rw + %u 2 bias ) 1/2 and expanded UM.  ... 
doi:10.1515/cclm-2015-5028 fatcat:lzpfzlp32rhn7cg2ymrjtmz34y

Decision-Making under Miscalibration [article]

Guy N. Rothblum, Gal Yona
2022 arXiv   pre-print
When the risk estimates are perfectly calibrated, the answer is well understood: a classification problem's cost structure induces an optimal treatment threshold j^⋆.  ...  We provide closed form expressions for j when miscalibration is measured using both expected and maximum calibration error, which reveal that it indeed differs from j^⋆ (the optimal threshold under perfect  ...  GY is also additionally supported by the Israeli Council for Higher Education (CHE) via the Weizmann Data Science Research Center, by a research grant from the Estate of Tully and Michele Plesser, and  ... 
arXiv:2203.09852v1 fatcat:y7oj4gvwxncw5cwutekbvrpyim

Hydrological behaviour through experimental and modelling approaches:application to the Haute-Mentue catchment

Daniela Balin
For the second response, the soil storage saturation deficit, the uncertainty after the multicalibration approach reduces significantly, most of the time the observed data being within the uncertainty  ...  MULTI-RESPONSE CALIBRATION: In order to assess the significance of the multicalibration approach, we present below the results concerning the predictive uncertainty for both discharge and soil storage  ...  How to go on with the field experimental approach… and which would be the most effective way to spend money on measurements for constraining the uncertainties in distributed model predictions?  ... 
doi:10.5075/epfl-thesis-3007 fatcat:rubdihbctjbl3kjkudbswmlmiy

Advances and Open Problems in Federated Learning [article]

Peter Kairouz, H. Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, Rafael G.L. D'Oliveira, Hubert Eichner (+47 others)
2021 arXiv   pre-print
Acknowledgments The authors would like to thank Alex Ingerman and David Petrou for their useful suggestions and insightful comments during the review process.  ...  The Bayesian approach further offers uncertainty estimates via its parameters in form of probability distributions, thus preventing over-fitting.  ...  From a probability theory perspective, it is unjustifiable to use single point-estimates for classification.  ... 
arXiv:1912.04977v3 fatcat:efkbqh4lwfacfeuxpe5pp7mk6a