Filters








18,525 Hits in 2.0 sec

Adversarial Evaluation of Dialogue Models [article]

Anjuli Kannan, Oriol Vinyals
2017 arXiv   pre-print
The recent application of RNN encoder-decoder models has resulted in substantial progress in fully data-driven dialogue systems, but evaluation remains a challenge.  ...  An adversarial loss could be a way to directly evaluate the extent to which generated dialogue responses sound like they came from a human.  ...  This work investigates the use of an adversarial evaluation method for dialogue models.  ... 
arXiv:1701.08198v1 fatcat:iftw37tyn5belh4pletdkqo3ja

An Adversarial Learning Framework For A Persona-Based Multi-Turn Dialogue Model

Oluwatobi Olabiyi, Anish Khazane, Alan Salimov, Erik Mueller
2019 Proceedings of the Workshop on Methods for Optimizing and Evaluating Neural Language Generation   unpublished
To demonstrate the superior performance of phredGAN over the persona Seq2Seq model, we experiment with two conversational datasets, the Ubuntu Dialogue Corpus (UDC) and TV series transcripts from the Big  ...  Performance comparison is made with respect to a variety of quantitative measures as well as crowd-sourced human evaluation.  ...  of dialogue response quality as well as automatic evaluations with perplexity, BLEU, ROUGE and distinct n-gram scores.  ... 
doi:10.18653/v1/w19-2301 fatcat:ayesb4xgonhafeckiwvjtzoh7u

Learning from Perturbations: Diverse and Informative Dialogue Generation with Inverse Adversarial Training [article]

Wangchunshu Zhou, Qifei Li, Chenle Li
2021 arXiv   pre-print
In this paper, we propose Inverse Adversarial Training (IAT) algorithm for training neural dialogue systems to avoid generic responses and model dialogue history better.  ...  In addition, we point out a problem of the widely used maximum mutual information (MMI) based methods for improving the diversity of dialogue response generation models and demonstrate it empirically.  ...  This suggests that our proposed IAT objective does not harm the adversarial robustness. Human evaluation We conduct a human evaluation of compared models on the DailyDialog dataset.  ... 
arXiv:2105.15171v1 fatcat:3mtnqftcgbh4jafy5chc66et2u

Neural Conversation Model Controllable by Given Dialogue Act Based on Adversarial Learning and Label-aware Objective

Seiya Kawano, Koichiro Yoshino, Satoshi Nakamura
2019 Proceedings of the 12th International Conference on Natural Language Generation  
Building a controllable neural conversation model (NCM) is an important task. In this paper, we focus on controlling the responses of NCMs by using dialogue act labels of responses as conditions.  ...  We introduce an adversarial learning framework for the task of generating conditional responses with a new objective to a discriminator, which explicitly distinguishes sentences by using labels.  ...  of each dialogue act (adversarial-explicit).  ... 
doi:10.18653/v1/w19-8627 dblp:conf/inlg/KawanoYN19 fatcat:rpdpwbzo2zdivna4hv2gqzthdi

Adversarial Learning for Neural Dialogue Generation [article]

Jiwei Li, Will Monroe, Tianlin Shi, Sébastien Jean, Alan Ritter and Dan Jurafsky
2017 arXiv   pre-print
In addition to adversarial training we describe a model for adversarial evaluation that uses success in fooling an adversary as a dialogue evaluation metric, while avoiding a number of potential pitfalls  ...  The outputs from the discriminator are then used as rewards for the generative model, pushing the system to generate dialogues that mostly resemble human dialogues.  ...  Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of DARPA, the NSF, or Facebook.  ... 
arXiv:1701.06547v5 fatcat:2rs2su3zjbaj5ethwgum4kb7nq

Adversarial Learning for Neural Dialogue Generation

Jiwei Li, Will Monroe, Tianlin Shi, Sėbastien Jean, Alan Ritter, Dan Jurafsky
2017 Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing  
In addition to adversarial training we describe a model for adversarial evaluation that uses success in fooling an adversary as a dialogue evaluation metric, while avoiding a number of potential pitfalls  ...  The outputs from the discriminator are then used as rewards for the generative model, pushing the system to generate dialogues that mostly resemble human dialogues.  ...  Acknowledgements The authors thank Michel Galley, Bill Dolan, Chris Brockett, Jianfeng Gao and other members of the NLP group at Mi-  ... 
doi:10.18653/v1/d17-1230 dblp:conf/emnlp/LiMSJRJ17 fatcat:c6cwxrxmyzajrha5d3pmp4rl3e

One "Ruler" for All Languages: Multi-Lingual Dialogue Evaluation with Adversarial Multi-Task Learning [article]

Xiaowei Tong, Zhenxin Fu, Mingyue Shang, Dongyan Zhao, Rui Yan
2018 arXiv   pre-print
Automatic evaluating the performance of Open-domain dialogue system is a challenging problem.  ...  We evaluate the proposed model in two different languages.  ...  The update of each Bi-LSTM unit can be Overview of adversarial multi-task neural metric for multi-lingual dialogue evaluation.  ... 
arXiv:1805.02914v1 fatcat:ju2j2sfflvattg3kowcd5yx5ly

One "Ruler" for All Languages: Multi-Lingual Dialogue Evaluation with Adversarial Multi-Task Learning

Xiaowei Tong, Zhenxin Fu, Mingyue Shang, Dongyan Zhao, Rui Yan
2018 Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence  
Automatic evaluating the performance of Open-domain dialogue system is a challenging problem.  ...  We evaluate the proposed model in two different languages.  ...  This work was supported by the National Key Research and Development Program of China (No. 2017YFC0804001), the National Science Foundation of China (No. 61672058).  ... 
doi:10.24963/ijcai.2018/616 dblp:conf/ijcai/TongFSZY18 fatcat:mzoaw4lbx5ghpg4nmuqrcnz3ku

Evaluating and Enhancing the Robustness of Dialogue Systems: A Case Study on a Negotiation Agent

Minhao Cheng, Wei Wei, Cho-Jui Hsieh
2019 Proceedings of the 2019 Conference of the North  
In this paper, we develop algorithms to evaluate the robustness of a dialogue agent by carefully designed attacks using adversarial agents.  ...  Furthermore, we demonstrate that adversarial training using our attacks can significantly improve the robustness of a goaloriented dialogue system.  ...  Conclusion In this paper, we develop adversarial agents to evaluate the robustness of a goal-oriented dialogue system.  ... 
doi:10.18653/v1/n19-1336 dblp:conf/naacl/ChengWH19 fatcat:6ac5b5tlhbex3c4v2rpklgso3i

Dynamic Knowledge Graph-based Dialogue Generation with Improved Adversarial Meta-Learning [article]

Hongcai Xu, Junpeng Bao, Gaojie Zhang
2020 arXiv   pre-print
KDAD formulates dynamic knowledge triples as a problem of adversarial attack and incorporates the objective of quickly adapting to dynamic knowledge-aware dialogue generation.  ...  However, these models do not take into account the sparseness and incompleteness of knowledge graph (KG)and current dialogue models cannot be applied to dynamic KG.  ...  We evaluate and show that our model outperforms the state-of-the-art baselines (Qadpt and TAware).  ... 
arXiv:2004.08833v1 fatcat:scp3yjftivcdzarc4zxn2266ee

Synthesizing Adversarial Negative Responses for Robust Response Ranking and Evaluation [article]

Prakhar Gupta, Yulia Tsvetkov, Jeffrey P. Bigham
2021 arXiv   pre-print
Open-domain neural dialogue models have achieved high performance in response ranking and evaluation tasks.  ...  We propose approaches for automatically creating adversarial negative training data to help ranking and evaluation models learn features beyond content similarity.  ...  Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the funding agencies.  ... 
arXiv:2106.05894v1 fatcat:hpesb4bivbfbzoyjrfu3u3urhe

Adversarial Over-Sensitivity and Over-Stability Strategies for Dialogue Models

Tong Niu, Mohit Bansal
2018 Proceedings of the 22nd Conference on Computational Natural Language Learning  
We present two categories of model-agnostic adversarial strategies that reveal the weaknesses of several generative, task-oriented dialogue models: Should-Not-Change strategies that evaluate over-sensitivity  ...  , requires only 1/4 of the original vocabulary size, and is robust to one of the adversarial strategies (to which the original model is vulnerable) even without adversarial training.  ...  The views contained in this article are those of the authors and not of the funding agency.  ... 
doi:10.18653/v1/k18-1047 dblp:conf/conll/NiuB18 fatcat:x7662x5t7rdohpo342fhxnsgmy

Dialogue Generation: From Imitation Learning to Inverse Reinforcement Learning [article]

Ziming Li, Julia Kiseleva, Maarten de Rijke
2018 arXiv   pre-print
The performance of adversarial dialogue generation models relies on the quality of the reward signal produced by the discriminator.  ...  We evaluate the performance of the resulting model with automatic metrics and human evaluations in two annotation settings.  ...  Context Next Reply from Speaker A Speaker A: we did a story on this guy . he owns half of arizona . Speaker B: is he a fraud ?  ... 
arXiv:1812.03509v1 fatcat:t52on5kbavh6jhhovi2bxo4xki

Ethical Challenges in Data-Driven Dialogue Systems [article]

Peter Henderson, Koustuv Sinha, Nicolas Angelard-Gontier, Nan Rosemary Ke, Genevieve Fried, Ryan Lowe, Joelle Pineau
2017 arXiv   pre-print
Here, we highlight potential ethical issues that arise in dialogue systems research, including: implicit biases in data-driven systems, the rise of adversarial examples, potential sources of privacy violations  ...  The use of dialogue systems as a medium for human-machine interaction is an increasingly prevalent paradigm.  ...  The authors then evaluated state-of-art Question and Answering (QA) models based on the adversarial accuracy of answers, and found that the performance of all models significantly dropped.  ... 
arXiv:1711.09050v1 fatcat:3s33w4ljeffxxomgj2kq5jjvo4

An Adversarially-Learned Turing Test for Dialog Generation Models [article]

Xiang Gao, Yizhe Zhang, Michel Galley, Bill Dolan
2021 arXiv   pre-print
The design of better automated dialogue evaluation metrics offers the potential of accelerate evaluation research on conversational AI.  ...  However, existing trainable dialogue evaluation models are generally restricted to classifiers trained in a purely supervised manner, which suffer a significant risk from adversarial attacking (e.g., a  ...  To learn a more robust evaluation metric, we propose to train a model to discriminate machine outputs from human outputs via iterative adversarial training, instead of training evaluation model with a  ... 
arXiv:2104.08231v1 fatcat:jihc6lhmdjdntpmjveto45qzz4
« Previous Showing results 1 — 15 out of 18,525 results