Learning Bias-reduced Word Embeddings Using Dictionary Definitions

Haozhe An, Xiaojiang Liu, Donald Zhang
2022 Findings of the Association for Computational Linguistics: ACL 2022   unpublished
Pre-trained word embeddings, such as GloVe, have shown undesirable gender, racial, and religious biases. To address this problem, we propose DD-GloVe, a train-time debiasing algorithm to learn word embeddings by leveraging dictionary definitions. We introduce dictionaryguided loss functions that encourage word embeddings to be similar to their relatively neutral dictionary definition representations. Existing debiasing algorithms typically need a precompiled list of seed words to represent the
more » ... ias direction, along which biased information gets removed. Producing this list involves subjective decisions and it might be difficult to obtain for some types of biases. We automate the process of finding seed words: our algorithm starts from a single pair of initial seed words and automatically finds more words whose definitions display similar attributes traits. We demonstrate the effectiveness of our approach with benchmark evaluations and empirical analyses. Our code is available at https://github. com/haozhe-an/DD-GloVe.
doi:10.18653/v1/2022.findings-acl.90 fatcat:lllcqpmb65gmdantwvtf2ec54y