Deep learning from "passive feeding" to "selective eating" of real-world data

Zhongwen Li, Chong Guo, Danyao Nie, Duoru Lin, Yi Zhu, Chuan Chen, Lanqin Zhao, Xiaohang Wu, Meimei Dongye, Fabao Xu, Chenjin Jin, Ping Zhang (+3 others)
2020 npj Digital Medicine  
Artificial intelligence (AI) based on deep learning has shown excellent diagnostic performance in detecting various diseases with good-quality clinical images. Recently, AI diagnostic systems developed from ultra-widefield fundus (UWF) images have become popular standard-of-care tools in screening for ocular fundus diseases. However, in real-world settings, these systems must base their diagnoses on images with uncontrolled quality ("passive feeding"), leading to uncertainty about their
more » ... nce. Here, using 40,562 UWF images, we develop a deep learning–based image filtering system (DLIFS) for detecting and filtering out poor-quality images in an automated fashion such that only good-quality images are transferred to the subsequent AI diagnostic system ("selective eating"). In three independent datasets from different clinical institutions, the DLIFS performed well with sensitivities of 96.9%, 95.6% and 96.6%, and specificities of 96.6%, 97.9% and 98.8%, respectively. Furthermore, we show that the application of our DLIFS significantly improves the performance of established AI diagnostic systems in real-world settings. Our work demonstrates that "selective eating" of real-world data is necessary and needs to be considered in the development of image-based AI systems.
doi:10.1038/s41746-020-00350-y pmid:33145439 pmcid:PMC7603327 fatcat:arm52bvgmrafzia6m3vyyb7g44