50,687 Hits in 5.4 sec

Language Model Transformers as Evaluators for Open-domain Dialogues

Rostislav Nedelchev, Jens Lehmann, Ricardo Usbeck
2020 Proceedings of the 28th International Conference on Computational Linguistics   unpublished
We demonstrate that human evaluators have a positive correlation between the output of the language models and scores.  ...  In this work, we investigate whether language models (LM) based on transformer neural networks can indicate the quality of a conversation.  ...  the German Federal Ministry of Education and Research (BMBF) projects and excellence clusters ML2R (FKZ 01 15 18038 A/B/C), MLwin (01S18050 D/F), ScaDS.AI (01/S18026A) as well as the Fraunhofer Zukunftsstiftung  ... 
doi:10.18653/v1/2020.coling-main.599 fatcat:uhcwyl5iw5byjkke7ej2gvb7i4

A Unified Pre-training Framework for Conversational AI [article]

Siqi Bao, Bingjin Chen, Huang He, Xin Tian, Han Zhou, Fan Wang, Hua Wu, Haifeng Wang, Wenquan Wu, Yingzhan Lin
2021 arXiv   pre-print
With superior capability on capturing one-to-many mapping, such models are suitable for the open-domain conversation and knowledge grounded dialogue.  ...  PLATO-2 is initially designed as an open-domain chatbot, trained via two-stage curriculum learning.  ...  Acknowledgments We would like to thank the reviewers for their constructive suggestions; Jingzhou He, and Tingting Li for the help on resource coordination; Gaopeng Yong, Liankai Huang, and Hua Lu for  ... 
arXiv:2105.02482v2 fatcat:2r626k7rp5aktdlmysdyygsnu4

Hello, It's GPT-2 – How Can I Help You? Towards the Use of Pretrained Language Models for Task-Oriented Dialogue Systems [article]

Paweł Budzianowski, Ivan Vulić
2019 arXiv   pre-print
Data scarcity is a long-standing and crucial challenge that hinders quick development of task-oriented dialogue systems across multiple domains: task-oriented dialogue models are expected to learn grammar  ...  ., 2019) and generative model pre-training (Radford et al., 2019), we validate the approach on complex multi-domain task-oriented dialogues from the MultiWOZ dataset.  ...  This has been validated also for open-domain dialogue modeling Golovanov et al., 2019) .  ... 
arXiv:1907.05774v2 fatcat:n2zjh5vgf5fsbpyr43mtnw74o4

Hello, It's GPT-2 - How Can I Help You? Towards the Use of Pretrained Language Models for Task-Oriented Dialogue Systems

Paweł Budzianowski, Ivan Vulić
2019 Proceedings of the 3rd Workshop on Neural Generation and Translation  
Data scarcity is a long-standing and crucial challenge that hinders quick development of task-oriented dialogue systems across multiple domains: task-oriented dialogue models are expected to learn grammar  ...  We propose a taskoriented dialogue model that operates solely on text input: it effectively bypasses explicit policy and language generation modules.  ...  This has been validated also for open-domain dialogue modeling Golovanov et al., 2019) .  ... 
doi:10.18653/v1/d19-5602 dblp:conf/emnlp/BudzianowskiV19 fatcat:7oq5npapb5ecdiulkhhjqrz5i4

Improvement of a dedicated model for open domain persona-aware dialogue generation [article]

Qiang Han
2020 arXiv   pre-print
The dedicated model studied here refers to the open domain persona-aware dialogue generation model, and the dataset is multi turn short dialogue, The total length of a single input sequence is no more  ...  Therefore, many improvements in the architecture and attention mechanism of transformer architecture for long sequence processing are not discussed in this paper.  ...  Open domain dialogue generation is an important and very complex task of NLP.  ... 
arXiv:2008.11970v1 fatcat:ewbhyb7b6jhtdljrfll4tnnoji

Evaluating Empathetic Chatbots in Customer Service Settings [article]

Akshay Agarwal, Shashank Maiya, Sonu Aggarwal
2021 arXiv   pre-print
Recent advances have demonstrated how open-domain chatbots can be trained to demonstrate empathy when responding to live human utterances.  ...  , than a model without such training.  ...  Authorship Statement Akshay Agarwal -identifying Twitter dataset, experiment structure, model evaluation code for key scenarios.  ... 
arXiv:2101.01334v1 fatcat:blx5uu2qlzczxjf4mjx5clbisu

Conversational Question Reformulation via Sequence-to-Sequence Architectures and Pretrained Language Models [article]

Sheng-Chieh Lin, Jheng-Hong Yang, Rodrigo Nogueira, Ming-Feng Tsai, Chuan-Ju Wang, Jimmy Lin
2020 arXiv   pre-print
In CQR benchmarks of task-oriented dialogue systems, we evaluate fine-tuned PLMs on the recently-introduced CANARD dataset as an in-domain task and validate the models using data from the TREC 2019 CAsT  ...  Track as an out-domain task.  ...  Additionally, we would like to thank Google for computational resources in the form of Google Cloud credits.  ... 
arXiv:2004.01909v1 fatcat:shda5tedrfbzponhnygs3pfwsa

Coral: An Approach for Conversational Agents in Mental Health Applications [article]

Harsh Sakhrani, Saloni Parekh, Shubham Mahajan
2021 arXiv   pre-print
To this effect, we present an approach for creating a generative empathetic open-domain chatbot that can be used for mental health applications.  ...  Our models achieve state-of-the-art results on the Empathetic Dialogues test set.  ...  The model is formulated as an auto-regressive language model but was pre-trained on large-scale dialogue pairs extracted from Reddit discussion chains.  ... 
arXiv:2111.08545v1 fatcat:hc4ba3e5bfcepbfq33lfcsnnoa

Are Pre-trained Language Models Knowledgeable to Ground Open Domain Dialogues? [article]

Yufan Zhao, Wei Wu, Can Xu
2020 arXiv   pre-print
We study knowledge-grounded dialogue generation with pre-trained language models.  ...  Instead of pursuing new state-of-the-art on benchmarks, we try to understand if the knowledge stored in parameters of the pre-trained models is already enough to ground open domain dialogues, and thus  ...  Overall, it seems that pre-trained language models can be used to ground open domain dialogues as long as we can find a few dialogues carrying knowledge for fine-tuning, though how to obtain such dialogues  ... 
arXiv:2011.09708v1 fatcat:idmn6xkcubh4ffmsnk6xxuyxnm

Neural Dialogue Generation Methods in Open Domain: A Survey

Bin Sun, Kan Li
2021 Natural Language Processing Research  
For example, Microsoft Xiaobing is currently the most famous open-domain dialogue system. This article mainly focuses on Open-Domain Dialogue System.  ...  A B S T R A C T Open-Domain Dialogue Generation (human-computer interaction) is an important issue in the field of Natural Language Processing (NLP).  ...  ACKNOWLEDGMENTS We are grateful to the anonymous reviewers for their valuable and constructional advices on the previous versions of this article; all remaining errors are our own.  ... 
doi:10.2991/nlpr.d.210223.001 fatcat:mqcjkf7vczfkdjhdtbznupmz2e

Beyond Goldfish Memory: Long-Term Open-Domain Conversation [article]

Jing Xu, Arthur Szlam, Jason Weston
2021 arXiv   pre-print
Despite recent improvements in open-domain dialogue models, state of the art models are trained and evaluated on short conversations with little context.  ...  We show how existing models trained on existing datasets perform poorly in this long-term conversation setting in both automatic and human evaluations, and we study long-context models that can perform  ...  Modeling Multi-Session Chat Transformer Encoder-Decoders The most straight-forward approach for modeling dialogue using our new task is simply to use a large language model as is standard in open-domain  ... 
arXiv:2107.07567v1 fatcat:wrabh7xfcba67n2nj6jfbqlesq

Better Automatic Evaluation of Open-Domain Dialogue Systems with Contextualized Embeddings [article]

Sarik Ghazarian, Johnny Tian-Zheng Wei, Aram Galstyan, Nanyun Peng
2019 arXiv   pre-print
Despite advances in open-domain dialogue systems, automatic evaluation of such systems is still a challenging problem.  ...  Traditional reference-based metrics such as BLEU are ineffective because there could be many valid responses for a given context that share no common words with reference responses.  ...  Acknowledgments We thank the anonymous reviewers for their constructive feedback, as well as the members of the PLUS lab for their useful discussion and feedback.  ... 
arXiv:1904.10635v1 fatcat:n3qzpwd3dzeejiau2qgy23zzfy

DynaEval: Unifying Turn and Dialogue Level Evaluation [article]

Chen Zhang, Yiming Chen, Luis Fernando D'Haro, Yan Zhang, Thomas Friedrichs, Grandee Lee, Haizhou Li
2021 arXiv   pre-print
Experiments show that DynaEval significantly outperforms the state-of-the-art dialogue coherence model, and correlates strongly with human judgements across multiple dialogue evaluation aspects at both  ...  To this end, we propose DynaEval, a unified automatic evaluation framework which is not only capable of performing turn-level evaluation, but also holistically considers the quality of the entire dialogue  ...  Acknowledgement We would like to thank all the anonymous reviewers for their constructive comments. This work is supported by Human-Robot Interaction Phase 1 (Grant No. 19225  ... 
arXiv:2106.01112v3 fatcat:ypfqamagybd55dsnlx465uo5jm

Profile Consistency Identification for Open-domain Dialogue Agents [article]

Haoyu Song, Yan Wang, Wei-Nan Zhang, Zhengyu Zhao, Ting Liu, Xiaojiang Liu
2020 arXiv   pre-print
Further evaluations on downstream tasks demonstrate that the profile consistency identification model is conducive for improving dialogue consistency.  ...  Maintaining a consistent attribute profile is crucial for dialogue agents to naturally converse with humans.  ...  We thank all the anonymous reviewers for their helpful comments and suggestions.  ... 
arXiv:2009.09680v3 fatcat:hvysl44s7veulfg7pzndu4uw4q

PLATO-XL: Exploring the Large-scale Pre-training of Dialogue Generation [article]

Siqi Bao, Huang He, Fan Wang, Hua Wu, Haifeng Wang, Wenquan Wu, Zhihua Wu, Zhen Guo, Hua Lu, Xinxian Huang, Xin Tian, Xinchao Xu (+2 others)
2021 arXiv   pre-print
To train such large models, we adopt the architecture of unified transformer with high computation and parameter efficiency.  ...  We further explore the capacity of PLATO-XL on other conversational tasks, such as knowledge grounded dialogue and task-oriented conversation.  ...  Luo, and Dou Hong for the assistance with infrastructure.  ... 
arXiv:2109.09519v1 fatcat:kma55sd5ifaszjhbfppp7dswzy
« Previous Showing results 1 — 15 out of 50,687 results