A Study of Representational Similarity: The Emergence of Object Concepts in Rapid Serial Visual Presentation Streams [article]

Ivy Xiao Dan Zhou, Tijl Grootswagers, Blake Segula, Amanda Robinson, Sophia Shatek, Christopher Whyte, Thomas Carlson
2019 bioRxiv   pre-print
Guided by the observation that similar words in language occur in similar contexts, linguistic computational models trained on statistics of word co-occurrence in texts were shown to be effective in modelling both human performance in psycholinguistic tasks and semantically imbued representations in the brain. But, it remains unclear whether the semantic representation extracted from the distributional behavior of words in the natural language resemble the knowledge that is primarily acquired
more » ... tside language, through sensory-perceptual experience. Using Representational Similarity Analysis (RSA), the present study endeavours to identify a direct link between the neural representation of object concepts and computational modelling. The broad aim of this study is to examine the extent to which neural representation of object concepts can be modelled by two types of linguistic computational models: distributional word co-occurrence and lexical hierarchical. The more specific aim of the study is to investigate which of the two types of semantic structure, distributional or hierarchical, best explains the time-varying neural representation of object concepts in the brain. Subsequently, this study first applied time-resolved Multivariate Pattern Analysis (MVPA) to neural responses evoked by naturalistic images portraying a broad and large (n = 1854) set of object concepts and decoded the four general concept categories: natural, animal, food and drink, and clothing. Then, using RSA, the study compared the geometric structure in the time-varying neural representation of object concept categories with the structure in semantic representations produced by the two broad types of linguistic models. Contrary to previous research results, this study shows evidence that the structure of time-varying neural representations of object concepts corresponds primarily with the hierarchical structure produce by WordNet models. But it also notes that despite their different conceptualizations of word meaning, all linguistic models showed similar correlation paths with the neural data. This thesis concludes that the temporal synchrony between the models coupled with the potential influence from non-hierarchical relations in WordNet suggest the rapid transition from perception to representation, compatible with language and conceptual thoughts, is underpinned by concept category distinctive features.
doi:10.1101/824565 fatcat:fa2uesxhlvcnxlxyd5pfn2ns5i