Filters








10 Hits in 2.8 sec

Span-ConveRT: Few-shot Span Extraction for Dialog with Pretrained Conversational Representations [article]

Sam Coope, Tyler Farghly, Daniela Gerz, Ivan Vulić, Matthew Henderson
2020 arXiv   pre-print
We introduce Span-ConveRT, a light-weight model for dialog slot-filling which frames the task as a turn-based span extraction task.  ...  We show that leveraging such knowledge in Span-ConveRT is especially useful for few-shot learning scenarios: we report consistent gains over 1) a span extractor that trains representations from scratch  ...  Acknowledgments We thank the three anonymous reviewers for their helpful suggestions and feedback.  ... 
arXiv:2005.08866v2 fatcat:2hqd5sukxzcxfp2sjrep5sb7ga

Span-ConveRT: Few-shot Span Extraction for Dialog with Pretrained Conversational Representations

Samuel Coope, Tyler Farghly, Daniela Gerz, Ivan Vulić, Matthew Henderson
2020 Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics   unpublished
We introduce Span-ConveRT, a light-weight model for dialog slot-filling which frames the task as a turn-based span extraction task.  ...  We show that leveraging such knowledge in Span-ConveRT is especially useful for few-shot learning scenarios: we report consistent gains over 1) a span extractor that trains representations from scratch  ...  Acknowledgments We thank the three anonymous reviewers for their helpful suggestions and feedback.  ... 
doi:10.18653/v1/2020.acl-main.11 fatcat:45c22xyu7vhipjsqvo3cunl52m

ConVEx: Data-Efficient and Few-Shot Slot Labeling [article]

Matthew Henderson, Ivan Vulić
2021 arXiv   pre-print
We report state-of-the-art performance of ConVEx across a range of diverse domains and data sets for dialog slot-labeling, with the largest gains in the most challenging, few-shot setups.  ...  We propose ConVEx (Conversational Value Extractor), an efficient pretraining and fine-tuning neural approach for slot-labeling dialog tasks.  ...  We would also like to thank Sam Coope and Tyler Farghly for their help with rerunning and validating Span-BERT and Span-ConveRT.  ... 
arXiv:2010.11791v2 fatcat:bx24kqkh3vb5dkhcwpumaonniq

GenSF: Simultaneous Adaptation of Generative Pre-trained Models and Slot Filling [article]

Shikib Mehri, Maxine Eskenazi
2021 arXiv   pre-print
GenSF achieves state-of-the-art results on two slot filling datasets with strong gains in few-shot and zero-shot settings. We achieve a 9 F1 score improvement in zero-shot slot filling.  ...  We present GenSF (Generative Slot Filling), which leverages a generative pre-trained open-domain dialog model for slot filling.  ...  In few-shot settings, we achieve a 30 F 1 score improvement over Span-BERT and Span-ConveRT.  ... 
arXiv:2106.07055v1 fatcat:lt5xtztkfvhotl7o56bha2jqcq

Soloist: BuildingTask Bots at Scale with Transfer Learning and Machine Teaching

Baolin Peng, Chunyuan Li, Jinchao Li, Shahin Shayandeh, Lars Liden, Jianfeng Gao
2021 Transactions of the Association for Computational Linguistics  
Experiments show that (i)Soloist creates new state-of-the-art on well-studied task-oriented dialog benchmarks, including CamRest676 and MultiWOZ; (ii) in the few-shot fine-tuning settings, Soloist significantly  ...  with the system.  ...  Span-ConveRT (Coope et al., 2020) extends the framework to entity extraction. SC-GPT uses a pre-trained language model to convert a dialog act to a natural language response.  ... 
doi:10.1162/tacl_a_00399 fatcat:66kfsqab5jdp3jr4btcn3btcby

DialogZoo: Large-Scale Dialog-Oriented Task Learning [article]

Zhi Chen, Jijia Bao, Lu Chen, Yuncong Liu, Da Ma, Bei Chen, Mengyue Wu, Su Zhu, Jian-Guang Lou, Kai Yu
2022 arXiv   pre-print
Building unified conversational agents has been a long-standing goal of the dialogue research community. Most previous works only focus on a subset of various dialogue tasks.  ...  The experimental results show that our method not only improves the ability of dialogue generation and knowledge distillation, but also the representation ability of models.  ...  These intuitive baselines are the corresponding PLMs while other baselines are pre-trained dialog models, including ConvBERT (Mehri et al., 2020) and Span-ConveRT (Coope et al., 2020b) , which are both  ... 
arXiv:2205.12662v1 fatcat:bjjvinmorzbljk7274juubbevm

DialoGLUE: A Natural Language Understanding Benchmark for Task-Oriented Dialogue [article]

Shikib Mehri, Mihail Eric, Dilek Hakkani-Tur
2020 arXiv   pre-print
Language Understanding Evaluation), a public benchmark consisting of 7 task-oriented dialogue datasets covering 4 distinct natural language understanding tasks, designed to encourage dialogue research in representation-based  ...  For all the few-shot experiments, we train five times with different random seeds and report the average performance across the five runs.  ...  The MLM pre-training and multi-tasking is performed with only the few-shot versions of each dataset.  ... 
arXiv:2009.13570v2 fatcat:tzcnyusgajgn5osewdyuiabczm

ConVEx: Data-Efficient and Few-Shot Slot Labeling

Matthew Henderson, Ivan Vulić
2021 Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies   unpublished
We report state-of-the-art performance of ConVEx across a range of diverse domains and data sets for dialog slot-labeling, with the largest gains in the most challenging, few-shot setups.  ...  We propose ConVEx (Conversational Value Extractor), an efficient pretraining and finetuning neural approach for slot-labeling dialog tasks.  ...  We would also like to thank Sam Coope and Tyler Farghly for their help with rerunning and validating Span-BERT and Span-ConveRT.  ... 
doi:10.18653/v1/2021.naacl-main.264 fatcat:sr6dvgjtjrcclkif3u4glhnsau

Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Survey [article]

Jinjie Ni, Tom Young, Vlad Pandelea, Fuzhao Xue, Erik Cambria
2022 arXiv   pre-print
Furthermore, we comprehensively review the evaluation methods and datasets for dialogue systems to pave the way for future research.  ...  We speculate that this work is a good starting point for academics who are new to the dialogue systems or those who want to quickly grasp up-to-date techniques in this area.  ...  Coope et al. (2020) proposed Span-ConveRT, which was a pretrained model designed for slot filling task.  ... 
arXiv:2105.04387v5 fatcat:yd3gqg45rjgzxbiwfdlcvf3pye

When does Further Pre-training MLM Help? An Empirical Study on Task-Oriented Dialog Pre-training

Qi Zhu, Yuxian Gu, Lingxiao Luo, Bing Li, Cheng Li, Wei Peng, Minlie Huang, Xiaoyan Zhu
2021 Proceedings of the Second Workshop on Insights from Negative Results in NLP   unpublished
Span-ConveRT: and Noah A. Smith. 2020. Don’t stop pretraining: Few-shot span extraction for dialog with pretrained Adapt language models to domains and tasks.  ...  In conversational representations.  ... 
doi:10.18653/v1/2021.insights-1.9 fatcat:leyye34clvb5dir437yk4opvre