Span-ConveRT: Few-shot Span Extraction for Dialog with Pretrained Conversational Representations

Samuel Coope, Tyler Farghly, Daniela Gerz, Ivan Vulić, Matthew Henderson
2020 Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics   unpublished
We introduce Span-ConveRT, a light-weight model for dialog slot-filling which frames the task as a turn-based span extraction task. This formulation allows for a simple integration of conversational knowledge coded in large pretrained conversational models such as Con-veRT (Henderson et al., 2019a). We show that leveraging such knowledge in Span-ConveRT is especially useful for few-shot learning scenarios: we report consistent gains over 1) a span extractor that trains representations from
more » ... ch in the target domain, and 2) a BERTbased span extractor. In order to inspire more work on span extraction for the slot-filling task, we also release RESTAURANTS-8K, a new challenging data set of 8,198 utterances, compiled from actual conversations in the restaurant booking domain.
doi:10.18653/v1/2020.acl-main.11 fatcat:45c22xyu7vhipjsqvo3cunl52m