A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is
Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)
The underlying difference of linguistic patterns between general text and task-oriented dialogue makes existing pre-trained language models less useful in practice. In this work, we unify nine human-human and multi-turn task-oriented dialogue datasets for language modeling. To better model dialogue behavior during pre-training, we incorporate user and system tokens into the masked language modeling. We propose a contrastive objective function to simulate the response selection task. Ourdoi:10.18653/v1/2020.emnlp-main.66 fatcat:lekm4zgpzbeo7ln24su3xwemia