Memory, Show the Way: Memory Based Few Shot Word Representation Learning

Jingyuan Sun, Shaonan Wang, Chengqing Zong
2018 Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing  
Distributional semantic models (DSMs) generally require sufficient examples for a word to learn a high quality representation. This is in stark contrast with human who can guess the meaning of a word from one or a few referents only. In this paper, we propose Mem2Vec, a memory based embedding learning method capable of acquiring high quality word representations from fairly limited context. Our method directly adapts the representations produced by a DSM with a longterm memory to guide its
more » ... y to guide its guess of a novel word. Based on a pre-trained embedding space, the proposed method delivers impressive performance on two challenging few-shot word similarity tasks. Embeddings learned with our method also lead to considerable improvements over strong baselines on NER and sentiment classification.
doi:10.18653/v1/d18-1173 dblp:conf/emnlp/SunWZ18 fatcat:skgqcydy2zbbnlmvjufwels6mq