The perceptual processing of fused multi-spectral imagery

Elizabeth L. Fox, Joseph W. Houpt
2016 Cognitive Research  
Multi-spectral imagery can enhance decision-making by supplying multiple complementary sources of information. However, overloading an observer with information can deter decision-making. Hence, it is critical to assess multi-spectral image displays using human performance. Accuracy and response times (RTs) are fundamental for assessment, although without sophisticated empirical designs, they offer little information about why performance is better or worse. Systems factorial technology (SFT)
more » ... a framework for study design and analysis that examines observers' processing mechanisms, not just overall performance. In the current work, we use SFT to compare a display with two sensor images alongside each another with a display in which there is a single composite image. In our first experiment, the SFT results indicated that both display approaches suffered from limited workload capacity and more so for the composite imagery. In the second experiment, we examined the change in observer performance over the course of multiple days of practice. Participants' accuracy and RTs improved with training, but their capacity limitations were unaffected. Using SFT, we found that the capacity limitation was not due to an inefficient serial examination of the imagery by the participants. There are two clear implications of these results: Observers are less efficient with multi-spectral images than single images and the side-by-side display of source images is a viable alternative to composite imagery. SFT was necessary for these conclusions because it provided an appropriate mechanism for comparing single-source images to multi-spectral images and because it ruled out serial processing as the source of the capacity limitation. Significance When information from two sensors is for the most part redundant, multi-sensor fusion hinders performance, regardless of whether the images are presented side-byside or fused into a single composite image. An observer may instead benefit from one single-sensor image that provides the requisite information to make an accurate quick decision. If images from both sensors are displayed, presenting the images side-by-side leads to less inefficient performance than algorithmically fused images. With the particular imagery we used, it is clear that the inefficient performance with the side-by-side imagery is not due to serial processing or to waiting for complete processing of both sources. Instead, the limitation is more likely due to attentional or other intrinsic limitations.
doi:10.1186/s41235-016-0030-7 pmid:28180181 pmcid:PMC5256484 fatcat:waqlpi3y6bgvxeoxnzqsvvzvo4