Validation of an Internationally Derived Patient Severity Phenotype to Support COVID-19 Analytics from Electronic Health Record Data

Jeffrey G Klann, Griffin M Weber, Hossein Estiri, Bertrand Moal, Paul Avillach, Chuan Hong, Victor Castro, Thomas Maulhardt, Amelia L M Tan, Alon Geva, Brett K Beaulieu-Jones, Alberto Malovini (+30 others)
<span title="2021-02-10">2021</span> <i title="Oxford University Press"> <a target="_blank" rel="noopener" href="" style="color: black;">JAMIA Journal of the American Medical Informatics Association</a> </i> &nbsp;
The Consortium for Clinical Characterization of COVID-19 by EHR (4CE) is an international collaboration addressing COVID-19 with federated analyses of electronic health record (EHR) data. We sought to develop and validate a computable phenotype for COVID-19 severity. Twelve 4CE sites participated. First we developed an EHR-based severity phenotype consisting of six code classes, and we validated it on patient hospitalization data from the 12 4CE clinical sites against the outcomes of ICU
more &raquo; ... on and/or death. We also piloted an alternative machine-learning approach and compared selected predictors of severity to the 4CE phenotype at one site. The full 4CE severity phenotype had pooled sensitivity of 0.73 and specificity 0.83 for the combined outcome of ICU admission and/or death. The sensitivity of individual code categories for acuity had high variability - up to 0.65 across sites. At one pilot site, the expert-derived phenotype had mean AUC 0.903 (95% CI: 0.886, 0.921), compared to AUC 0.956 (95% CI: 0.952, 0.959) for the machine-learning approach. Billing codes were poor proxies of ICU admission, with as low as 49% precision and recall compared to chart review. We developed a severity phenotype using 6 code classes that proved resilient to coding variability across international institutions. In contrast, machine-learning approaches may overfit hospital-specific orders. Manual chart review revealed discrepancies even in the gold-standard outcomes, possibly due to heterogeneous pandemic conditions. We developed an EHR-based severity phenotype for COVID-19 in hospitalized patients and validated it at 12 international sites.
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="">doi:10.1093/jamia/ocab018</a> <a target="_blank" rel="external noopener" href="">pmid:33566082</a> <a target="_blank" rel="external noopener" href="">pmcid:PMC7928835</a> <a target="_blank" rel="external noopener" href="">fatcat:lef5r7lnsjfyposxkee3di5nn4</a> </span>
<a target="_blank" rel="noopener" href="" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href=""> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> </button> </a> <a target="_blank" rel="external noopener" href="" title="pubmed link"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> </button> </a>