Looking Beyond Label Noise: Shifted Label Distribution Matters in Distantly Supervised Relation Extraction [article]

Qinyuan Ye, Liyuan Liu, Maosen Zhang, Xiang Ren
2019 arXiv   pre-print
In recent years there is a surge of interest in applying distant supervision (DS) to automatically generate training data for relation extraction (RE). In this paper, we study the problem what limits the performance of DS-trained neural models, conduct thorough analyses, and identify a factor that can influence the performance greatly, shifted label distribution. Specifically, we found this problem commonly exists in real-world DS datasets, and without special handing, typical DS-RE models
more » ... t automatically adapt to this shift, thus achieving deteriorated performance. To further validate our intuition, we develop a simple yet effective adaptation method for DS-trained models, bias adjustment, which updates models learned over the source domain (i.e., DS training set) with a label distribution estimated on the target domain (i.e., test set). Experiments demonstrate that bias adjustment achieves consistent performance gains on DS-trained models, especially on neural models, with an up to 23 data can be found at .
arXiv:1904.09331v2 fatcat:676knvbzyjcovbh7y3lxsdl7fe