How to Train good Word Embeddings for Biomedical NLP

Billy Chiu, Gamal Crichton, Anna Korhonen, Sampo Pyysalo
2016 Proceedings of the 15th Workshop on Biomedical Natural Language Processing  
The quality of word embeddings depends on the input corpora, model architectures, and hyper-parameter settings. Using the state-of-the-art neural embedding tool word2vec and both intrinsic and extrinsic evaluations, we present a comprehensive study of how the quality of embeddings changes according to these features. Apart from identifying the most influential hyper-parameters, we also observe one that creates contradictory results between intrinsic and extrinsic evaluations. Furthermore, we
more » ... Furthermore, we find that bigger corpora do not necessarily produce better biomedical domain word embeddings. We make our evaluation tools and resources as well as the created state-of-the-art word embeddings available under open licenses from https://github.com/ cambridgeltl/BioNLP-2016.
doi:10.18653/v1/w16-2922 dblp:conf/bionlp/ChiuCKP16 fatcat:vejh35fuqngwbp6ukyammmzfcu