A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Filters
ROC and AUC with a Binary Predictor: a Potentially Misleading Metric
[article]
2019
arXiv
pre-print
Preserved Fulltext
Other Versions
Overall, we believe a linear interpolation from the ROC curve with binary predictors, which is most commonly done in software, corresponding to the estimated AUC. ...
We believe these ROC curves and AUC can lead to misleading results. We compare R, Python, Stata, and SAS software implementations. ...
Although many discuss the properties of ROC and AUC analyses, we wish to first show the math and calculations of the AUC with a binary predictor and we then explore commonly-used statistical software for ...
arXiv:1903.04881v1
fatcat:x3cei2qgffhebavyi3mxe2whle
ROC plot and AUC with binary classifiers: pragmatic analysis of cognitive screening instruments
[article]
2021
medRxiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
It has been suggested that ROC and AUC may be potentially misleading when examining binary predictors rather than continuous scales. ...
The purpose of this study was to examine ROC plots and AUC values for two binary classifiers of cognitive status (applause sign, attended with sign), a cognitive screening instrument producing categorical ...
Hence the use of AUC is a potentially misleading metric in . ...
doi:10.1101/2021.03.09.21253194
fatcat:2ugpx4mwjraavadi4kbq4poqdy
Random Forests, Decision Trees, and Categorical Predictors: The "Absent Levels" Problem
[article]
2018
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Furthermore, by using three real data examples, we illustrate how absent levels can dramatically alter a model's performance in practice, and we empirically demonstrate how some simple heuristics can be ...
Although these incidents may appear to be innocuous, by using Leo Breiman and Adele Cutler's random forests FORTRAN code and the randomForest R package (Liaw and Wiener, 2002) as motivating case studies ...
The author would also like to thank David Chan, Robert Bell, the action editor, and the anonymous reviewers for their helpful feedback. ...
arXiv:1706.03492v2
fatcat:3fcncob2xvdsnncnk2wer3rxmi
Attention to issues and facts
2019
The Agenda Setting Journal
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
A new method is introduced that combines survey data with users' Facebook accounts and their actual Facebook posts to reveal the historical news sharing behaviors of 741 U.S. citizens. ...
Results suggest that a key component found in need for orientation – attention to relevant issues and facts – predicts observed political news sharing on Facebook. ...
My understanding of ROC analysis has benefited greatly from discussions with him. ...
doi:10.1075/asj.18004.var
fatcat:lf64je33uzhj5bsiy2j53n3j3q
Comparison of missing data handling methods for variant pathogenicity predictors
[article]
2022
bioRxiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
These solutions rely on the comprehensiveness of the available data and struggle with the sparse nature of genetic variant data. ...
Various machine learning (ML), artificial intelligence (AI), and in silico variant pathogenicity predictors have been developed to solve this challenge. ...
We also present results with the area under ROC curve (AUC-ROC, or just AUC) metric, defined as the area under the receiver operating characteristic curve. ...
doi:10.1101/2022.06.17.496578
fatcat:cgkyrxs6cneydoc5fa5xviok64
ROC curves for clinical prediction models part 1: ROC plots showed no added value above the AUC when evaluating the performance of clinical prediction models
2020
Journal of Clinical Epidemiology
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Classification plots can be supplemented with measures such as net benefit to assess the potential clinical utility of a model. ...
Receiver operating characteristic (ROC) curves are widely used in reports on clinical risk prediction models to demonstrate the ability of a model to discriminate between patients with and without a certain ...
Acknowledgments We acknowledge English language editing by Dr Jennifer A de Beyer of the Center for Statistics in Medicine, University of Oxford. ...
doi:10.1016/j.jclinepi.2020.01.028
pmid:32712176
fatcat:etp3eseudjd3xnq2qupblnhssy
Calibrating variant-scoring methods for clinical decision making
2021
Bioinformatics
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Poorly calibrated algorithms can be misleading and potentially harmful for clinical decision-making. Supplementary information Supplementary data are available at Bioinformatics online. ...
Calibration refers to the idea that if a model predicts a group of variants to be pathogenic with a probability P, it is expected that the same fraction P of true positive is found in the observed set. ...
AUC performance of the different predictors One of the most commonly used metrics for assessing the performance of the classification methods is the AUC-ROC ( An ideal classifier would have an AUC-ROC ...
doi:10.1093/bioinformatics/btaa943
pmid:33492342
pmcid:PMC8023678
fatcat:ynitweouergs5li46hecnssoxi
Large-scale comparative assessment of computational predictors for lysine post-translational modification sites
2018
Briefings in Bioinformatics
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
With this goal in mind, we first provide a comprehensive survey on a large collection of 49 state-of-the-art approaches for lysine PTM prediction. ...
We cover a variety of important aspects that are crucial for the development of successful predictors, including operating algorithms, sequence and structural features, feature selection, model performance ...
ROC curves and AUC values of MUscADEL and other predictors for (A) acetylation, (B) glycation, (C) malonylation, (D) methylation, (E) succinylation, (F) sumoylation and (G) ubiquitination. ...
doi:10.1093/bib/bby089
pmid:30285084
pmcid:PMC6954452
fatcat:ipqhpnlhufegnodxthnwl75t4y
Binary classifier metrics for optimizing HEP event selection
2019
EPJ Web of Conferences
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
I discuss the choice of evaluation metrics for binary classifiers in High Energy Physics (HEP) event selection and I point out that the Area Under the ROC Curve (AUC) is of limited relevance in this context ...
I propose new metrics based on Fisher information, which can be used for both the evaluation and training of HEP event selection algorithms in statistically limited measurements of a parameter. ...
The main problem with the AUC, both in medical diagnostics and in concrete applications of ML techniques in banking and commerce, is that it is misleading when comparing two ROCs that cross: the AUC is ...
doi:10.1051/epjconf/201921406004
fatcat:p5lrs6jiirco7j6j6e2lmvsqra
Multi-temporal analysis reveals that predictors of mountain pine beetle infestation change during outbreak cycles
2013
Forest Ecology and Management
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2013; you can also visit the original URL.
The file type is application/pdf.
In early years, red attack was associated with forest and topographic chara cteristics known to influence susceptibility to MPB. ...
algorithm with manually selected training classes. ...
Negrón and an anonymous reviewer for comments that improved the manuscript. ...
doi:10.1016/j.foreco.2013.03.038
fatcat:n4ga66m2vnhmbpcmdehq2zo2xi
The importance of prediction model validation and assessment in obesity and nutrition research
2015
International Journal of Obesity
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
In this overview, we explain why reductions in predictive validity occur, define the metrics commonly used to estimate the predictive validity of a model (for example, coefficient of determination (R 2 ...
, and adjusted and shrunken R 2 ). ...
ACKNOWLEDGEMENTS This study was supported in part by NIH grants R25DK099080, R25HL124208 and P30DK056336. ...
doi:10.1038/ijo.2015.214
pmid:26449421
pmcid:PMC4826636
fatcat:fjzfccp2yfgltkqjeshpb3aq3u
Partner‐specific Prediction of RNA‐binding Residues in Proteins: A Critical Assessment
2018
Proteins: Structure, Function, and Bioinformatics
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Specifically, we introduce a novel metric, RNA-specificity metric (RSM), for quantifying the RNA-specificity of the RNA binding residues predicted by such tools. ...
In this work, we assess the performance of two recently published partner-specific protein-RNA interface prediction tools, PS-PRIP, and PRIdictor, along with our own new tools. ...
, AUC [ROC], and AUC [CROC], with differences in any metric (if any) at most 2%. ...
doi:10.1002/prot.25639
pmid:30536635
pmcid:PMC6389706
fatcat:t22qizjhzvblxdtkz6xd5iwcai
Caveats and pitfalls of ROC analysis in clinical microarray research (and how to avoid them)
2011
Briefings in Bioinformatics
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL.
The file type is application/pdf.
ROC curves are frequently summarized in a single scalar, the area under the curve (AUC). ...
This article discusses the caveats and pitfalls of ROC analysis in clinical microarray research, particularly in relation to (i) the interpretation of AUC (especially a value close to 0.5); (ii) model ...
To avoid potentially misleading interpretations, it is important to acknowledge the fundamental difference between classification and ranking. ROC measures the latter. ...
doi:10.1093/bib/bbr008
pmid:21422066
fatcat:oki4d65ounhxrburkogfdclaga
Who has a Choice?: Survey-Based Predictors of Volitionality in Facebook Use and Non-use
2021
Proceedings of the ACM on Human-Computer Interaction
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Furthermore, they provide potential implications both for future work and for technology policy. ...
, a sense of their own agency, and, across both studies, level of education. ...
IIS-1421498 and Grant No. CNS-1814533) and the Facebook Computational Social Science Methodology Research Award. Thanks to our survey participants for sharing their data and their experiences. ...
doi:10.1145/3463935
fatcat:sdhfmr5yzvhtrpdsq6oc6qddcm
Benchmarking machine learning models for the analysis of genetic data using FRESA.CAD Binary Classification Benchmarking
[article]
2019
bioRxiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
However, with the availability of a wide variety of such methods, model selection has become increasingly difficult, both from the human and computational perspective. ...
FRESA.CAD Binary Benchmarking demonstrates to be a useful tool over a variety of binary classification problems comprising the analysis of genetic data showing both quantitative and qualitative advantages ...
the Bioinformatics for Clinical Diagnosis research program, Escuela de Medicina y Ciencias de la Salud, Tecnologico de Monterrey for their valuable comments during the coding of the FRESA.CAD package and ...
doi:10.1101/733675
fatcat:gyfatv5etrhermewf6goy7uqsa
« Previous
Showing results 1 — 15 out of 804 results