Filters








173 Hits in 7.6 sec

Adversarial Training for Multi-task and Multi-lingual Joint Modeling of Utterance Intent Classification

Ryo Masumura, Yusuke Shinohara, Ryuichiro Higashinaka, Yushi Aono
2018 Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing  
This paper proposes an adversarial training method for the multi-task and multi-lingual joint modeling needed for utterance intent classification.  ...  The effectiveness of the adversarial training proposal is demonstrated using Japanese and English data sets for three different utterance intent classification tasks.  ...  Proposed Method This section details our adversarial training method for multi-task and multi-lingual joint modeling of utterance intent classification.  ... 
doi:10.18653/v1/d18-1064 dblp:conf/emnlp/MasumuraSHA18 fatcat:qjgaqsy24rge7bk6dfiwmymtzi

Multi-task and Multi-lingual Joint Learning of Neural Lexical Utterance Classification based on Partially-shared Modeling

Ryo Masumura, Tomohiro Tanaka, Ryuichiro Higashinaka, Hirokazu Masataki, Yushi Aono
2018 International Conference on Computational Linguistics  
This paper is an initial study on multi-task and multi-lingual joint learning for lexical utterance classification.  ...  We demonstrate the effectiveness of the proposed adversarial training using Japanese and English data sets with three different lexical utterance classification tasks.  ...  Multi-task and Multi-lingual Neural Lexical Utterance Classification This section presents multi-task and multi-lingual joint learning of neural lexical utterance classification.  ... 
dblp:conf/coling/MasumuraTHMA18 fatcat:edno24wierhwvcbguvo7re67yy

Call Larisa Ivanovna: Code-Switching Fools Multilingual NLU Models [article]

Alexey Birshert, Ekaterina Artemova
2021 arXiv   pre-print
In such setup models for cross-lingual transfer show remarkable performance in joint intent recognition and slot filling.  ...  The evaluation of NLU models seems biased and limited, since code-switching is being left out of scope.  ...  We show-case that monolingual models fail to process code-switched utterances. At the same time, cross-lingual models cope much better with such texts; 3.  ... 
arXiv:2109.14350v2 fatcat:lx4uuiloffegde25ebnbb6utru

The Impact of Semi-Supervised Learning on the Performance of Intelligent Chatbot System

Sudan Prasad Uprety, Seung Ryul Jeong
2022 Computers Materials & Continua  
the predictive performance of the proposed multi-domain intelligent chatbot over a base joint model and joint model based on adversarial learning.  ...  Systematic experimental analysis of the proposed joint frameworks, along with the semi-supervised multi-domain model, using open-source annotated and unannotated utterances shows robust improvement in  ...  A single semi-supervised multi-domain joint model (SEMI-MDJM) based on LSTM outperforms a joint base model and an adversarial multi-domain joint model in each task i.e., domain classification, intent prediction  ... 
doi:10.32604/cmc.2022.023127 fatcat:mohe2hpgpzdmjdxwz32wiu4vtm

Cross-Lingual Multi-Task Neural Architecture for Spoken Language Understanding

Yujiang Li, Xuemin Zhao, Weiqun Xu, Yonghong Yan
2018 Interspeech 2018  
We first investigate a joint model of slot filling and intent determination for SLU, which alleviates the outof-vocabulary problem and explicitly models dependencies between output labels by combining  ...  Cross-lingual spoken language understanding (SLU) systems traditionally require machine translation services for language portability and liberation from human supervision.  ...  Multi-task and cross-lingual training Multi-task training with attention For joint modeling of intent determination and slot filling, additional intent classification layer shares the same word-level  ... 
doi:10.21437/interspeech.2018-1039 dblp:conf/interspeech/LiZX018 fatcat:2hv45jpcx5dhhjhorfshfrwik4

To What Degree Can Language Borders Be Blurred In BERT-based Multilingual Spoken Language Understanding? [article]

Quynh Do, Judith Gaspers, Tobias Roding, Melanie Bradford
2020 arXiv   pre-print
In addition, we propose a novel BERT-based adversarial model architecture to learn language-shared and language-specific representations for multilingual SLU.  ...  Our experimental results prove that the proposed model is capable of narrowing the gap to the ideal multilingual performance.  ...  The model is trained by optimizing the joint loss L = L i + L s where L i and L s are the cross entropy loss for intent identification and the CRF loss for slot classification, respectively.  ... 
arXiv:2011.05007v1 fatcat:f2tfdj4av5giff5r64v7bw4tqe

A Survey on Spoken Language Understanding: Recent Advances and New Frontiers [article]

Libo Qin, Tianbao Xie, Wanxiang Che, Ting Liu
2021 arXiv   pre-print
With the burst of deep neural networks and the evolution of pre-trained language models, the research of SLU has obtained significant breakthroughs.  ...  , implicit joint modeling vs. explicit joint modeling in joint model, non pre-trained paradigm vs. pre-trained paradigm;(2) new frontiers: some emerging areas in complex SLU as well as the corresponding  ...  Single Model Implicit Joint Modeling: Implicit joint modeling denotes Single models train each task separately for intent detection  ... 
arXiv:2103.03095v2 fatcat:krhrfeomafd6nds2m4o5djbzby

Multi-Task Learning in Natural Language Processing: An Overview [article]

Shijie Chen, Yu Zhang, Qiang Yang
2021 arXiv   pre-print
Then we present optimization techniques on loss construction, data sampling, and task scheduling to properly train a multi-task model.  ...  However, deep neural models often suffer from overfitting and data scarcity problems that are pervasive in NLP tasks.  ...  In [79] , a partially shared multi-task model is built for language intent learning through dialogue act classification, extended named entity classification, and question type classification in Japanese  ... 
arXiv:2109.09138v1 fatcat:hlgzjykuvzczzmsgnl32w5qo5q

A survey of joint intent detection and slot-filling models in natural language understanding [article]

H. Weld, X. Huang, S. Long, J. Poon, S. C. Han
2021 arXiv   pre-print
However, more recently, joint models for intent classification and slot filling have achieved state-of-the-art performance, and have proved that there exists a strong relationship between the two tasks  ...  Intent classification and slot filling are two critical tasks for natural language understanding. Traditionally the two tasks have been deemed to proceed independently.  ...  [Masumura et al. 2018] proposed an adversarial training method for the multi-task and multi-lingual joint modelling to improve performance on minority data.  ... 
arXiv:2101.08091v3 fatcat:ai6w2imilrfupf4m5fm2rjtzxi

XeroAlign: Zero-Shot Cross-lingual Transformer Alignment [article]

Milan Gritta, Ignacio Iacobacci
2021 arXiv   pre-print
XLM-RA's text classification accuracy exceeds that of XLM-R trained with labelled data and performs on par with state-of-the-art models on a cross-lingual adversarial paraphrasing task.  ...  We introduce XeroAlign, a simple method for task-specific alignment of cross-lingual pretrained transformers such as XLM-R.  ...  For text classification tasks such as Cross-lingual Natural Language Inference (Conneau et al., 2018) , an adversarial cross-lingual alignment was proposed by Qi and Du (2020) .  ... 
arXiv:2105.02472v2 fatcat:wnttyap5wbblncujqf3nquoeqi

Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systems [article]

Evgeniia Razumovskaia, Goran Glavaš, Olga Majewska, Edoardo M. Ponti, Anna Korhonen, Ivan Vulić
2022 arXiv   pre-print
We find that the most critical factor preventing the creation of truly multilingual ToD systems is the lack of datasets in most languages for both training and evaluation.  ...  To overcome this limitation, we draw parallels between components of the ToD pipeline and other NLP tasks, which can inspire solutions for learning in low-resource scenarios.  ...  Joint multi-task training, besides potentially reducing the number of parameters, is advantageous for NLU (Zhang et al., 2019b) , as the two tasks are clearly interdependent: intuitively, the slots that  ... 
arXiv:2104.08570v3 fatcat:a6vfgcvgqvhllfkgfwcr3mptgq

Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systems

Evgeniia Razumovskaia, Goran Glavas, Olga Majewska, Edoardo M. Ponti, Anna Korhonen, Ivan Vulic
2022 The Journal of Artificial Intelligence Research  
We find that the most critical factor preventing the creation of truly multilingual ToD systems is the lack of datasets in most languages for both training and evaluation.  ...  To overcome this limitation, we draw parallels between components of the ToD pipeline and other NLP tasks, which can inspire solutions for learning in low-resource scenarios.  ...  Joint multi-task training, besides potentially reducing the number of parameters, is advantageous for NLU (Zhang et al., 2019b) , as the two tasks are clearly interdependent: intuitively, the slots that  ... 
doi:10.1613/jair.1.13083 fatcat:54a6w62wxvbvvigh32zkmtwwqq

Locale-agnostic Universal Domain Classification Model in Spoken Language Understanding

Jihwan Lee, Ruhi Sarikaya, Young-Bum Kim
2019 Proceedings of the 2019 Conference of the North  
We propose a locale-agnostic universal domain classification model based on selective multi-task learning that learns a joint representation of an utterance over locales with different sets of domains  ...  The experimental results demonstrate the effectiveness of our approach on domain classification task in the scenario of multiple locales with imbalanced data and disparate domain sets.  ...  For future work, we consider adopting the proposed model architecture to multi-lingual scenario as well.  ... 
doi:10.18653/v1/n19-2002 dblp:conf/naacl/LeeSK19 fatcat:bgwljbbk5fg2jjdfguxker3dpq

GL-CLeF: A Global-Local Contrastive Learning Framework for Cross-lingual Spoken Language Understanding [article]

Libo Qin, Qiguang Chen, Tianbao Xie, Qixin Li, Jian-Guang Lou, Wanxiang Che, Min-Yen Kan
2022 arXiv   pre-print
, and semantic-level Global transfer across intent and slot).  ...  In addition, a key step in GL-CLeF is a proposed Local and Global component, which achieves a fine-grained cross-lingual transfer (i.e., sentence-level Local intent transfer, token-level Local slot transfer  ...  This work was supported by the National Key R&D Program of China via grant 2020AAA0106501 and the National Natural Science Foundation of China (NSFC) via grant 61976072 and 62176078.  ... 
arXiv:2204.08325v1 fatcat:r4xxcj6hg5bixbnogrisq7fvh4

Multi-Level Cross-Lingual Transfer Learning with Language Shared and Specific Knowledge for Spoken Language Understanding

Keqing He, Weiran Xu, Yuanmeng Yan
2020 IEEE Access  
Given these properties of the task, the problem can be stated as a joint sentence classification (for intent classification) and slot tagging (for slot detection) task.  ...  The state-ofthe-art models [4] , [5] guarantee exceptionally high accuracy on the tasks of slot tagging and intent classification under the availability of large-scale annotated datasets.  ... 
doi:10.1109/access.2020.2972925 fatcat:qz6yelorrjdx7ndr6si6x52r4y
« Previous Showing results 1 — 15 out of 173 results