Deep learning based retrieval system for gigapixel histopathology cases and open access literature [article]

Sebastian Otalora, Roger Schaer, Manfredo Atzori, Oscar Alfonso Jimenez del Toro, Henning Muller
2018 bioRxiv   pre-print
Clinical practice is getting increasingly stressful for pathologists due to increasing complexity and time constraints. Histopathology is slowly shifting to digital pathology, thus creating opportunities to allow pathologists to improve reading quality or save time using Artificial Intelligence (AI)-based applications. We aim to enhance the practice of pathologists through a retrieval system that allows them to simplify their workflow, limit the need for second opinions, while also learning in
more » ... he process. In this work, an innovative retrieval system for digital pathology is integrated within a Whole Slide Image (WSI) histopathology viewer, allowing to define regions of interest in images as queries for finding visually similar areas using deep learning representations. The back-end similarity computation algorithms are based on a multimodal approach, thus allowing to exploit both text information and content-based image features. Shallow and deep representations of the images were evaluated, the later showed a better overall retrieval performance in a set of 112 whole slide images from biopsies. The system was tested by pathologists, highlighting its capabilities and suggesting possible ways to improve it and make it more usable in clinical practice. The retrieval system developed can enhance the practice of pathologists by enabling them to use their experience and knowledge to properly control artificial intelligence tools for navigating repositories of images for decision support purposes.
doi:10.1101/408237 fatcat:nnlbgydgcrdizng2wp2rnc3fwq