Ontological attention ensembles for capturing semantic concepts in ICD code prediction from clinical text

Matus Falis, Maciej Pajak, Aneta Lisowska, Patrick Schrempf, Lucas Deckers, Shadia Mikhael, Sotirios Tsaftaris, Alison O'Neil
2019 Proceedings of the Tenth International Workshop on Health Text Mining and Information Analysis (LOUHI 2019)  
We present a semantically interpretable system for automated ICD coding of clinical text documents. Our contribution is an ontological attention mechanism which matches the structure of the ICD ontology, in which shared attention vectors are learned at each level of the hierarchy, and combined into label-dependent ensembles. Analysis of the attention heads shows that shared concepts are learned by the lowest common denominator node. This allows child nodes to focus on the differentiating
more » ... s, leading to efficient learning and memory usage. Visualisation of the multilevel attention on the original text allows explanation of the code predictions according to the semantics of the ICD ontology. On the MIMIC-III dataset we achieve a 2.7% absolute (11% relative) improvement from 0.218 to 0.245 macro-F1 score compared to the previous state of the art across 3,912 codes. Finally, we analyse the labelling inconsistencies arising from different coding practices which limit performance on this task.
doi:10.18653/v1/d19-6220 dblp:conf/acl-louhi/FalisPLSDMTO19 fatcat:w6gk5fyhfrhhppkxgxegq5jaia