A Two-stage Model for Slot Filling in Low-resource Settings: Domain-agnostic Non-slot Reduction and Pretrained Contextual Embeddings

Cennet Oguz, Ngoc Thang Vu
2020 Proceedings of SustaiNLP: Workshop on Simple and Efficient Natural Language Processing   unpublished
Learning-based slot filling -a key component of spoken language understanding systemstypically requires a large amount of in-domain hand-labeled data for training. In this paper, we propose a novel two-stage model architecture that can be trained with only a few indomain hand-labeled examples. The first step is designed to remove non-slot tokens (i.e., O labeled tokens), as they introduce noise in the input of slot filling models. This step is domain-agnostic and therefore, can be trained by
more » ... loiting out-of-domain data. The second step identifies slot names only for slot tokens by using state-of-the-art pretrained contextual embeddings such as ELMO and BERT. We show that our approach outperforms other state-of-art systems on the SNIPS benchmark dataset.
doi:10.18653/v1/2020.sustainlp-1.10 fatcat:wb3kctcpg5blzcqa2um4lmnyxa