Leveraging Preposition Ambiguity to Assess Compositional Distributional Models of Semantics

Samuel Ritter, Cotie Long, Denis Paperno, Marco Baroni, Matthew Botvinick, Adele Goldberg
2015 Proceedings of the Fourth Joint Conference on Lexical and Computational Semantics  
Complex interactions among the meanings of words are important factors in the function that maps word meanings to phrase meanings. Recently, compositional distributional semantics models (CDSM) have been designed with the goal of emulating these complex interactions; however, experimental results on the effectiveness of CDSM have been difficult to interpret because the current metrics for assessing them do not control for the confound of lexical information. We present a new method for
more » ... the degree to which CDSM capture semantic interactions that dissociates the influences of lexical and compositional information. We then provide a dataset for performing this type of assessment and use it to evaluate six compositional models using both co-occurrence based and neural language model input vectors. Results show that neural language input vectors are consistently superior to co-occurrence based vectors, that several CDSM capture substantial compositional information, and that, surprisingly, vector addition matches and is in many cases superior to purpose-built paramaterized models.
doi:10.18653/v1/s15-1023 dblp:conf/starsem/RitterLPBBG15 fatcat:g3sfbbzawzfalosec6otrkraim