Internet Archive Scholar

Naturalness Evaluation of Natural Language Generation in Task-oriented Dialogues using BERT

Ye Liu, Mercedes-Benz AG, Sindelfingen, Germany, Wolfgang Maier, Wolfgang Minker, Stefan Ultes, Mercedes-Benz AG, Sindelfingen, Germany, Ulm University, Ulm, Germany, Mercedes-Benz AG, Sindelfingen, Germany
2021 Proceedings of the Conference Recent Advances in Natural Language Processing - Deep Learning for Natural Language Processing Methods and Applications   unpublished

Preserved Fulltext

fulltext thumbnail
Web Archive Capture PDF (240.1 kB)
https://web.archive.org/web/20211117212514/https://acl-bg.org/proceedings/2021/RANLP%202021/pdf/2021.ranlp-1.96.pdf

A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL. The file type is application/pdf.

This paper presents an automatic method to evaluate the naturalness of natural language generation in dialogue systems. While this task was previously rendered through expensive and time-consuming human labor, we present this novel task of automatic naturalness evaluation of generated language. By fine-tuning the BERT model, our proposed naturalness evaluation method shows robust results and outperforms the baselines: support vector machines, bi-directional LSTMs, and BLEURT. In addition, the
more » ... aining speed and evaluation performance of naturalness model are improved by transfer learning from quality and informativeness linguistic knowledge.
doi:10.26615/978-954-452-072-4_096 fatcat:m2kbvccdufef7dg4lllhthe35m
Citation

Ye Liu, Mercedes-Benz AG, Sindelfingen, Germany, Wolfgang Maier, Wolfgang Minker, Stefan Ultes, Mercedes-Benz AG, Sindelfingen, Germany, Ulm University, Ulm, Germany, Mercedes-Benz AG, Sindelfingen, Germany. "Naturalness Evaluation of Natural Language Generation in Task-oriented Dialogues using BERT." Proceedings of the Conference Recent Advances in Natural Language Processing - Deep Learning for Natural Language Processing Methods and Applications (2021)