Filters








12,898 Hits in 6.3 sec

Learning to Customize Model Structures for Few-shot Dialogue Generation Tasks [article]

Yiping Song, Zequn Liu, Wei Bi, Rui Yan, Ming Zhang
2020 arXiv   pre-print
In this paper, we propose an algorithm that can customize a unique dialogue model for each task in the few-shot setting.  ...  However, fine-tuning distinguishes tasks from the parameter perspective but ignores the model-structure perspective, resulting in similar dialogue models for different tasks.  ...  Hence, this is the focus of our paper -few-shot dialogue generation, i.e. training a generative model that can be generalized to a new task (domain) within k-shots of its dialogues.  ... 
arXiv:1910.14326v2 fatcat:p4hxppee3nbjlk7cddwgbqange

Learning to Customize Model Structures for Few-shot Dialogue Generation Tasks

YIPING SONG, Zequn Liu, Wei Bi, Rui Yan, Ming Zhang
2020 Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics   unpublished
In this paper, we propose an algorithm that can customize a unique dialogue model for each task in the few-shot setting.  ...  However, fine-tuning distinguishes tasks from the parameter perspective but ignores the model-structure perspective, resulting in similar dialogue models for different tasks.  ...  Hence, this is the focus of our paper -few-shot dialogue generation, i.e. training a generative model that can be generalized to a new task (domain) within k-shots of its dialogues.  ... 
doi:10.18653/v1/2020.acl-main.517 fatcat:x3asl27qwffu5oyihvyvyi3pga

Coffee With a Hint of Data: Towards Using Data-Driven Approaches in Personalised Long-Term Interactions

Bahar Irfan, Mehdi Hellou, Tony Belpaeme
2021 Frontiers in Robotics and AI  
In addition, it is desirable to learn user preferences from a few samples of interactions (i.e., few-shot learning).  ...  The experiments show that while data-driven approaches are suitable for generic task-oriented dialogue and real-time interactions, no model performs sufficiently well to be deployed in personalised long-term  ...  ACKNOWLEDGMENTS The authors would like to thank Ghent University IDLab (Belgium) for access to the servers for the experiments, and Cité Internationale Universitaire de Paris (France) for hosting the user  ... 
doi:10.3389/frobt.2021.676814 pmid:34651017 pmcid:PMC8505524 fatcat:wxtxjfcuzvctdkrhyay2nemnyu

UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models [article]

Tianbao Xie, Chen Henry Wu, Peng Shi, Ruiqi Zhong, Torsten Scholak, Michihiro Yasunaga, Chien-Sheng Wu, Ming Zhong, Pengcheng Yin, Sida I. Wang, Victor Zhong, Bailin Wang (+11 others)
2022 arXiv   pre-print
UnifiedSKG also facilitates the investigation of zero-shot and few-shot learning, and we show that T0, GPT-3, and Codex struggle in zero-shot and few-shot learning for SKG.  ...  We also use UnifiedSKG to conduct a series of controlled experiments on structured knowledge encoding variants across SKG tasks.  ...  We thank Qian Liu for his TAPEX code and advice on question answering tasks. We thank wandb for free logging and OpenAI for free Codex usage.  ... 
arXiv:2201.05966v2 fatcat:veo3m24nizfvzohenlav26i6bu

Few-Shot Bot: Prompt-Based Learning for Dialogue Systems [article]

Andrea Madotto, Zhaojiang Lin, Genta Indra Winata, Pascale Fung
2021 arXiv   pre-print
In this paper, we explore prompt-based few-shot learning in dialogue tasks.  ...  The current largest released LM (GPT-J-6B) using prompt-based few-shot learning, and thus requiring no training, achieves competitive performance to fully trained state-of-the-art models.  ...  To cope with these issues, we propose to leverage prompt-based few-shot learning for the skill selection task.  ... 
arXiv:2110.08118v1 fatcat:fhcmp7x34ndh5cr253w44vvide

Attention Guided Dialogue State Tracking with Sparse Supervision [article]

Shuailong Liang, Lahari Poddar, Gyuri Szarvas
2021 arXiv   pre-print
The model learns a slot-aware representation of dialogue history, which focuses on relevant turns to guide the decoder.  ...  In call centers, for tasks like managing bookings or subscriptions, the user goal can be associated with actions (e.g.~API calls) issued by customer service agents.  ...  This helps to better initialize the state generator and makes the generation task much easier to learn.  ... 
arXiv:2101.11958v1 fatcat:vvlxgoaxcrdqdnjoflhvhntynu

ProQA: Structural Prompt-based Pre-training for Unified Question Answering [article]

Wanjun Zhong, Yifan Gao, Ning Ding, Yujia Qin, Zhiyuan Liu, Ming Zhou, Jiahai Wang, Jian Yin, Nan Duan
2022 arXiv   pre-print
Through a structurally designed prompt-based input schema, ProQA concurrently models the knowledge generalization for all QA tasks while keeping the knowledge customization for every specific QA task.  ...  The specialty in QA research hinders systems from modeling commonalities between tasks and generalization for wider applications.  ...  EM Step we first adapt ProQA to task A by few-shot learning to obtain the model A: f A θ with performance s A . Then we sequentially adapt f A θ to task B and receive task B model f AB θ .  ... 
arXiv:2205.04040v1 fatcat:b6kl7zvc7rgc3d6shubgfbxjbq

Integrating Pre-trained Model into Rule-based Dialogue Management [article]

Jun Quan, Meng Yang, Qiang Gan, Deyi Xiong, Yiming Liu, Yuchen Dong, Fangxin Ouyang, Jun Tian, Ruiling Deng, Yongzhi Li, Yang Yang, Daxin Jiang
2021 arXiv   pre-print
Then we propose the "model-trigger" design to make the DM trainable thus scalable to scenario changes. Furthermore, we integrate pre-trained models and empower the DM with few-shot capability.  ...  Rule-based dialogue management is still the most popular solution for industrial task-oriented dialogue systems for their interpretablility.  ...  We would also like to thank the anonymous reviewers for their insightful comments.  ... 
arXiv:2102.08553v1 fatcat:lvkvovokhvdczakhgqyxpj2nzy

Dialogue Generation on Infrequent Sentence Functions via Structured Meta-Learning [article]

Yifan Gao, Piji Li, Wei Bi, Xiaojiang Liu, Michael R. Lyu, Irwin King
2020 arXiv   pre-print
We treat dialogue generation conditioned on different sentence functions as separate tasks, and apply model-agnostic meta-learning to high-resource sentence functions data.  ...  In this paper, we investigate a structured meta-learning (SML) approach for dialogue generation on infrequent sentence functions.  ...  Recently, model-agnostic metalearning (MAML) (Finn et al., 2017) has shown promising results on several few-shot classification tasks.  ... 
arXiv:2010.01495v1 fatcat:uhnxk3uuhfennkdo2wle5didzm

Teaching Models new APIs: Domain-Agnostic Simulators for Task Oriented Dialogue [article]

Moya Chen, Paul A. Crook, Stephen Roller
2021 arXiv   pre-print
We demonstrate that large language models are able to simulate Task Oriented Dialogues in novel domains, provided only with an API implementation and a list of goals.  ...  Furthermore, by checking for whether the User's goals are met, we can use simulation to repeatedly generate training data and improve the quality of simulations themselves.  ...  Acknowledgements Thank you to members of the Facebook dialogue teams for their helpful feedback and suggestions on this project.  ... 
arXiv:2110.06905v1 fatcat:pigqvwkrgve4he435wshvxzh6e

AllWOZ: Towards Multilingual Task-Oriented Dialog Systems for All [article]

Lei Zuo, Kun Qian, Bowen Yang, Zhou Yu
2021 arXiv   pre-print
Furthermore, we create a benchmark for our multilingual dataset by applying mT5 with meta-learning.  ...  This paper presents AllWOZ, a multilingual multi-domain task-oriented customer service dialog dataset covering eight languages: English, Mandarin, Korean, Vietnamese, Hindi, French, Portuguese, and Thai  ...  We find that our model, which uses meta-learning to learn the shared structures between languages, performs significantly better than normal training in a few-shot setting and could achieve comparable  ... 
arXiv:2112.08333v1 fatcat:2q7klyeptbd6nijdk3c5jrw2ra

Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systems [article]

Evgeniia Razumovskaia, Goran Glavaš, Olga Majewska, Edoardo M. Ponti, Anna Korhonen, Ivan Vulić
2021 arXiv   pre-print
In task-oriented dialogue (ToD), a user holds a conversation with an artificial agent to complete a concrete task.  ...  To overcome this limitation, we draw parallels between components of the ToD pipeline and other NLP tasks, which can inspire solutions for learning in low-resource scenarios.  ...  fully zero-shot transfer (Lauscher et al., 2020) ; 2) the actual source language(s) for zero-shot and few-shot cross-lingual transfer in low-resource scenarios may have a huge impact on the final task  ... 
arXiv:2104.08570v2 fatcat:bi5xizz4wzct5fpiuk3ikotjta

ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation [article]

Yu Sun, Shuohuan Wang, Shikun Feng, Siyu Ding, Chao Pang, Junyuan Shang, Jiaxiang Liu, Xuyi Chen, Yanbin Zhao, Yuxiang Lu, Weixin Liu, Zhihua Wu (+10 others)
2021 arXiv   pre-print
It fuses auto-regressive network and auto-encoding network, so that the trained model can be easily tailored for both natural language understanding and generation tasks with zero-shot learning, few-shot  ...  Particularly, the GPT-3 model with 175 billion parameters shows its strong task-agnostic zero-shot/few-shot learning capabilities.  ...  The proposed ERNIE 3.0 can handle both natural language understanding tasks and natural language generation tasks through zero-shot learning, few-shot learning or fine-tuning.  ... 
arXiv:2107.02137v1 fatcat:uuocxl66cbhvtc7kkys3hpbgbu

Description-Driven Task-Oriented Dialog Modeling [article]

Jeffrey Zhao, Raghav Gupta, Yuan Cao, Dian Yu, Mingqiu Wang, Harrison Lee, Abhinav Rastogi, Izhak Shafran, Yonghui Wu
2022 arXiv   pre-print
Task-oriented dialogue (TOD) systems are required to identify key information from conversations for the completion of given tasks.  ...  This can lead to models memorizing arbitrary patterns in data, resulting in suboptimal performance and generalization.  ...  It is therefore natural to consider leveraging these models for few-shot dialogue modeling.  ... 
arXiv:2201.08904v1 fatcat:wzzw6xc4o5cp5ndxmoir4upv64

Dialogue Summaries as Dialogue States (DS2), Template-Guided Summarization for Few-shot Dialogue State Tracking [article]

Jamin Shin, Hangyeol Yu, Hyeongdon Moon, Andrea Madotto, Juneyoung Park
2022 arXiv   pre-print
Annotating task-oriented dialogues is notorious for the expensive and difficult data collection process. Few-shot dialogue state tracking (DST) is a realistic solution to this problem.  ...  To elaborate, we train a text-to-text language model with synthetic template-based dialogue summaries, generated by a set of rules from the dialogue states.  ...  We would also like to thank Zhaonjiang Lin for the helpful discussions.  ... 
arXiv:2203.01552v1 fatcat:gfrwg6mmzzbl7dyurtdd2pfk7q
« Previous Showing results 1 — 15 out of 12,898 results