Variation of SNOMED CT Coding of Clinical Research Concepts among Coding Experts

J. E. Andrews, R. L. Richesson, J. Krischer
2007 JAMIA Journal of the American Medical Informatics Association  
A b s t r a c t Objective: To compare consistency of coding among professional SNOMED CT coders representing three commercial providers of coding services when coding clinical research concepts with SNOMED CT. Design: A sample of clinical research questions from case report forms (CRFs) generated by the NIH-funded Rare Disease Clinical Research Network (RDCRN) were sent to three coding companies with instructions to code the core concepts using SNOMED CT. The sample consisted of 319
more » ... wer pairs from 15 separate studies. The companies were asked to select SNOMED CT concepts (in any form, including post-coordinated) that capture the core concept(s) reflected in the question. Also, they were asked to state their level of certainty, as well as how precise they felt their coding was. Measurements: Basic frequencies were calculated to determine raw level agreement among the companies and other descriptive information. Krippendorff's alpha was used to determine a statistical measure of agreement among the coding companies for several measures (semantic, certainty, and precision). Results: No significant level of agreement among the experts was found. Conclusion: There is little semantic agreement in coding of clinical research data items across coders from 3 professional coding services, even using a very liberal definition of agreement. Ⅲ J Am Med Inform Assoc.
doi:10.1197/jamia.m2372 pmid:17460128 pmcid:PMC2244907 fatcat:x344ia7lsbdpfb5lwywh4pssq4