Computational discrimination between natural images based on gaze during mental imagery

Xi Wang, Andreas Ley, Sebastian Koch, James Hays, Kenneth Holmqvist, Marc Alexa
2020 Scientific Reports  
When retrieving image from memory, humans usually move their eyes spontaneously as if the image were in front of them. Such eye movements correlate strongly with the spatial layout of the recalled image content and function as memory cues facilitating the retrieval procedure. However, how close the correlation is between imagery eye movements and the eye movements while looking at the original image is unclear so far. In this work we first quantify the similarity of eye movements between
more » ... ng an image and encoding the same image, followed by the investigation on whether comparing such pairs of eye movements can be used for computational image retrieval. Our results show that computational image retrieval based on eye movements during spontaneous imagery is feasible. Furthermore, we show that such a retrieval approach can be generalized to unseen images.
doi:10.1038/s41598-020-69807-0 pmid:32747683 fatcat:i5yzidmp4vhsrefgjuivbsxhd4