Second-order contexts from lexical substitutes for few-shot learning of word representations

Qianchu Liu, Diana McCarthy, Anna Korhonen
2019 Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (*  
There is a growing awareness of the need to handle rare and unseen words in word representation modelling. In this paper, we focus on few-shot learning of emerging concepts that fully exploits only a few available contexts. We introduce a substitute-based context representation technique that can be applied on an existing word embedding space. Previous context-based approaches to modelling unseen words only consider bag-of-word firstorder contexts, whereas our method aggregates contexts as
more » ... d-order substitutes that are produced by a sequence-aware sentence completion model. We experimented with three tasks that aim to test the modelling of emerging concepts. We found that these tasks show different emphasis on first and second order contexts, and our substitute-based method achieved superior performance on naturallyoccurring contexts from corpora.
doi:10.18653/v1/s19-1007 dblp:conf/starsem/LiuMK19 fatcat:3sdyzbzzgff3fgujoh7la2tg44