Inter-Method Performance Study of Tumor Volumetry Assessment on Computed Tomography Test-Retest Data

Andrew J. Buckler, Jovanna Danagoulian, Kjell Johnson, Adele Peskin, Marios A. Gavrielides, Nicholas Petrick, Nancy A. Obuchowski, Hubert Beaumont, Lubomir Hadjiiski, Rudresh Jarecha, Jan-Martin Kuhnigk, Ninad Mantri (+10 others)
2015 Academic Radiology  
Thirty-one lung cancer test-retest cases were analyzed by twelve participants in a multi-site study of algorithm performance on the segmentation of clinical CT scans. We evaluated variability of scalar volume measurements, including individual participant performance across test-retest repetitions, as well as the performance across algorithms. We also compared segmentation boundaries relative to reference standard segmentations. We report the repeatability of measurements in test-retest cases
more » ... r each participant, and reproducibility of measurements across participants. Repeatability coefficients (RC) ranged from .06 (best performing) to 1.5 (least performing), corresponding to within-subject coefficients of variation of 2.1% to 54% respectively. Reproducibility coefficient values (RDC) are somewhat greater than the lowest performing pooled participant's repeatability value. The best algorithm performance is seen when measured tumors meet the measurability criterion defined in the QIBA Profile; including tumors that did not resulted in approximately 1-1/2 times the variability. The value of editing segmentation results was equivocal; smaller tumors appear to be better without editing, but larger tumors benefit by editing. Linear mixed effects modelling was used to conclude that no more than two-thirds of the overall QIBA Profile variability claim of the system as a whole be allocated to analysis software if the overall system is to be compliant (or less if conditions such as the scanner settings are not held constant). An important outcome of this work is that the set of metrics used for this analysis form a basis for future determination of compliance with the QIBA Profile. Inter-method Performance Study of Tumor Volumetry Assessment on Computed Tomography Test-retest Data
doi:10.1016/j.acra.2015.08.007 pmid:26376841 pmcid:PMC4609285 fatcat:xxw7tmdr4fay5ky4zwjh5w4n44