Is BERT Robust to Label Noise? A Study on Learning with Noisy Labels in Text Classification [article]

Dawei Zhu, Michael A. Hedderich, Fangzhou Zhai, David Ifeoluwa Adelani, Dietrich Klakow
2022 arXiv   pre-print
Incorrect labels in training data occur when human annotators make mistakes or when the data is generated via weak or distant supervision. It has been shown that complex noise-handling techniques - by modeling, cleaning or filtering the noisy instances - are required to prevent models from fitting this label noise. However, we show in this work that, for text classification tasks with modern NLP models like BERT, over a variety of noise types, existing noisehandling methods do not always
more » ... its performance, and may even deteriorate it, suggesting the need for further investigation. We also back our observations with a comprehensive analysis.
arXiv:2204.09371v1 fatcat:ovdlsfhegvg45eqxrjj33iw6eq