Using Neural Networks to Extend Cropped Medical Images for Deformable Registration Among Images with Differing Scan Extents

Elizabeth M McKenzie, Nuo Tong, Dan Ruan, Minsong Cao, Robert K Chin, Ke Sheng
2021 Medical Physics (Lancaster)  
Missing or discrepant imaging volumes is a common challenge in deformable image registration (DIR). To minimize the adverse impact, we train a neural network to synthesize cropped portions of head and neck CT's and then test its use in DIR. Using a training dataset of 409 head and neck CT's, we trained a generative adversarial network to take in a cropped 3D image and output an image with synthesized anatomy in the cropped region. The network used a 3D U-Net generator along with VGG deep
more » ... losses. To test our technique, for each of the 53 test volumes, we used Elastix to deformably register combinations of a randomly cropped, full, and synthetically full volume to a single cropped, full, and synthetically full target volume. We additionally tested our method's robustness to crop extent by progressively increasing the amount of cropping, synthesizing the missing anatomy using our network, then performing the same registration combinations. Registration performance was measured using 95% Hausdorff distance across 16 contours. We successfully trained a network to synthesize missing anatomy in superiorly and inferiorly cropped images. The network can estimate large regions in an incomplete image, far from the cropping boundary. Registration using our estimated full images was not significantly different from registration using the original full images. The average contour matching error for full image registration was 9.9mm, while our method was 11.6mm, 12.1mm, and 13.6mm for synthesized-to-full, full-to-synthesized, and synthesized-to-synthesized registrations, respectively. In comparison, registration using the cropped images had errors of 31.7mm and higher. Plotting the registered image contour error as a function of initial pre-registered error shows that our method is robust to registration difficulty. Synthesized-to-full registration was statistically independent of cropping extent up to 18.7cm superiorly cropped. Synthesized-to-synthesized registration was nearly independent, with a -0.04mm change in average contour error for every additional millimeter of cropping. Different or inadequate in scan extent is a major cause of DIR inaccuracies. We address this challenge by training a neural network to complete cropped 3D images. We show that with image completion, the source of DIR inaccuracy is eliminated, and the method is robust to varying crop extent.
doi:10.1002/mp.15039 pmid:34101198 pmcid:PMC8683602 fatcat:k3hcn7flkzd6jidpkqapeo7kwq