Automatic generating the clinically accurate radiology report from X-ray images is important but challenging. The identification of multigrained abnormal regions in image and corresponding abnormalities is difficult for datadriven neural models. In this work, we introduce a Memory-aligned Knowledge Graph (MaKG) of clinical abnormalities to better learn the visual patterns of abnormalities and their relationships by integrating it into a deep model architecture for the report generation. We

more »

... out extensive experiments and show that the proposed MaKG deep model can improve the clinical accuracy of the generated reports.