Filters








171 Hits in 3.9 sec

Towards Incremental Transformers: An Empirical Analysis of Transformer Models for Incremental NLU [article]

Patrick Kahardipraja, Brielen Madureira, David Schlangen
2021 arXiv   pre-print
In this work, we examine the feasibility of LT for incremental NLU in English.  ...  Our results show that the recurrent LT model has better incremental performance and faster inference speed compared to the standard Transformer and LT with restart-incrementality, at the cost of part of  ...  Acknowledgements We thank the anonymous reviewers for their critical reading of our manuscript and their insightful comments and suggestions.  ... 
arXiv:2109.07364v1 fatcat:3xvfbw5qa5bunnefcfp2okuu6u

Towards Incremental Transformers: An Empirical Analysis of Transformer Models for Incremental NLU

Patrick Kahardipraja, Brielen Madureira, David Schlangen
2021 Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing   unpublished
In this work, we examine the feasibility of LT for incremental NLU in English.  ...  Recent work attempts to apply Transformers incrementally via restart-incrementality by repeatedly feeding, to an unchanged model, increasingly longer input prefixes to produce partial outputs.  ...  Acknowledgements We thank the anonymous reviewers for their critical reading of our manuscript and their insightful comments and suggestions.  ... 
doi:10.18653/v1/2021.emnlp-main.90 fatcat:npzzuy6z7bdgnn4gtwddclk6pa

IsoBN: Fine-Tuning BERT with Isotropic Batch Normalization [article]

Wenxuan Zhou, Bill Yuchen Lin, Xiang Ren
2021 arXiv   pre-print
This simple yet effective fine-tuning method yields about 1.0 absolute increment on the average of seven NLU tasks.  ...  Fine-tuning pre-trained language models (PTLMs), such as BERT and its better variant RoBERTa, has been a common practice for advancing performance in natural language understanding (NLU) tasks.  ...  towards more isotropic representations, yielding an absolute increment around 1.0 point on 7 popular NLU tasks.  ... 
arXiv:2005.02178v2 fatcat:qrapp4bnzraythbjlollhpoef4

Graph-to-Graph Transformer for Transition-based Dependency Parsing [article]

Alireza Mohammadshahi, James Henderson
2020 arXiv   pre-print
After proposing two novel Transformer models of transition-based dependency parsing as strong baselines, we show that adding the proposed mechanisms for conditioning on and predicting graphs of Graph2Graph  ...  We propose the Graph2Graph Transformer architecture for conditioning on and predicting arbitrary graphs, and apply it to the challenging task of transition-based dependency parsing.  ...  We also thank Lesly Miculicich, other members of the IDIAP NLU group, and anonymous reviewers for helpful discussions.  ... 
arXiv:1911.03561v4 fatcat:gn3k24ydhvbdrmqg77lu6gtxky

ClarET: Pre-training a Correlation-Aware Context-To-Event Transformer for Event-Centric Generation and Classification [article]

Yucheng Zhou, Tao Shen, Xiubo Geng, Guodong Long, Daxin Jiang
2022 arXiv   pre-print
In this paper, we propose to pre-train a general Correlation-aware context-to-Event Transformer (ClarET) for event-centric reasoning.  ...  The proposed ClarET is applicable to a wide range of event-centric reasoning scenarios, considering its versatility of (i) event-correlation types (e.g., causal, temporal, contrast), (ii) application formulations  ...  Natural Language Understanding (NLU). Our basic model, BART-large, is presented for general NLU tasks.  ... 
arXiv:2203.02225v2 fatcat:dt4kivmupngujhwgv53e3p2lci

Current Challenges in Spoken Dialogue Systems and Why They Are Critical for Those Living with Dementia [article]

Angus Addlesee, Arash Eshghi, Ioannis Konstas
2019 arXiv   pre-print
In this paper, we outline some of the challenges that are in urgent need of further research, including Incremental Speech Recognition and a systematic study of the interactional patterns in conversation  ...  Dialogue technologies such as Amazon's Alexa have the potential to transform the healthcare industry.  ...  A systematic, empirical study of such patterns is essential for informing design of dialogue systems for this target group -see Sec. 3.2.  ... 
arXiv:1909.06644v1 fatcat:ewmuxf7mhnasrmp4x2cihnvmny

Incremental Processing in the Age of Non-Incremental Encoders: An Empirical Assessment of Bidirectional Models for Incremental NLU [article]

Brielen Madureira, David Schlangen
2020 arXiv   pre-print
We test five models on various NLU datasets and compare their performance using three incremental evaluation metrics.  ...  The results support the possibility of using bidirectional encoders in incremental mode while retaining most of their non-incremental quality.  ...  Acknowledgments We are thankful to the three anonymous reviewers and to our colleagues in the Foundations of Computational Linguistics Lab at the University of Potsdam for their valuable comments and insights  ... 
arXiv:2010.05330v1 fatcat:wgspmk6ggze5voselhkqs26gca

ISIS

Helen Meng, P. C. Ching, Shuk Fong Chan, Yee Fong Wong, Cheong Chat Chan
2004 ACM Transactions on Computer-Human Interaction  
ISIS provides a system test-bed for our work in multilingual speech recognition and generation, speaker authentication, language understanding and dialog modeling.  ...  ISIS (Intelligent Speech for Information Systems) is a trilingual spoken dialog system (SDS) for the stocks domain.  ...  We thank the past and present members of the ISIS team from the Human-Computer Communications Laboratory and Digital Signal Processing Laboratory of CUHK, as well as the National Key Laboratory for Machine  ... 
doi:10.1145/1017494.1017497 fatcat:hdiorq5t7nhlxgjw3um2oydcla

Non-Local Part-Aware Point Cloud Denoising [article]

Chao Huang, Ruihui Li, Xianzhi Li, Chi-Wing Fu
2020 arXiv   pre-print
To enhance the denoising performance, we cascade a series of NLUs to progressively distill the noise features from the noisy inputs.  ...  Further, besides the conventional surface reconstruction loss, we formulate a semantic part loss to regularize the predictions towards the relevant parts and enable denoising in a part-aware manner.  ...  an incremental ablation study.  ... 
arXiv:2003.06631v1 fatcat:o7gjmwbdxndj5btadq37ite6uu

Sequence Package Analysis and Soft Computing: Introducing a New Hybrid Method to Adjust to the Fluid and Dynamic Nature of Human Speech [chapter]

Amy Neustein
2011 Advances in Intelligent and Soft Computing  
may transform a simple question into a rhetorical one; or transform an otherwise simple and straightforward assessment into a gratuitous/sardonic remark) cannot be adequately addressed by conventional  ...  natural language speech interface, neural networks, or connectionist models, may be viewed as the natural choice for investigating the patterns underlying the orderliness of talk, as they are equipped  ...  Perhaps this is the reason for the sea change in attitude toward acceptance of soft computing methods?  ... 
doi:10.1007/978-3-642-19644-7_1 fatcat:fehnwumlyja65fr3gtweb7773e

Detecting Egregious Conversations between Customers and Virtual Agents [article]

Tommy Sandbank, Michal Shmueli-Scheuer, Jonathan Herzig, David Konopnicki, John Richards, David Piorkowski
2018 arXiv   pre-print
In this paper, we outline an approach to detecting such egregious conversations, using behavioral cues from the user, patterns in agent responses, and user-agent interaction.  ...  Using logs of two commercial systems, we show that using these features improves the detection F1-score by around 20% over using textual features alone.  ...  Feature Set Contribution Analysis To better understand the contributions of different sets of features to our EGR model, we examined various features in an incremental fashion.  ... 
arXiv:1711.05780v2 fatcat:5kzker2bqjckzjdi3wkklakjua

Detecting Egregious Conversations between Customers and Virtual Agents

Tommy Sandbank, Michal Shmueli-Scheuer, Jonathan Herzig, David Konopnicki, John Richards, David Piorkowski
2018 Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)  
In this paper, we outline an approach to detecting such egregious conversations, using behavioral cues from the user, patterns in agent responses, and useragent interaction.  ...  Using logs of two commercial systems, we show that using these features improves the detection F1-score by around 20% over using textual features alone.  ...  Feature Set Contribution Analysis To better understand the contributions of different sets of features to our EGR model, we examined various features in an incremental fashion.  ... 
doi:10.18653/v1/n18-1163 dblp:conf/naacl/SandbankSHKRP18 fatcat:vuppzshnxbedtffscd3y5kjke4

ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation [article]

Yu Sun, Shuohuan Wang, Shikun Feng, Siyu Ding, Chao Pang, Junyuan Shang, Jiaxiang Liu, Xuyi Chen, Yanbin Zhao, Yuxiang Lu, Weixin Liu, Zhihua Wu (+10 others)
2021 arXiv   pre-print
In addition, most large-scale models are trained in an auto-regressive way.  ...  Empirical results show that the model outperforms the state-of-the-art models on 54 Chinese NLP tasks, and its English version achieves the first place on the SuperGLUE benchmark (July 3, 2021), surpassing  ...  Transformer-XL is similar to Transformer but introduces an auxiliary recurrence memory module to help modelling longer texts.  ... 
arXiv:2107.02137v1 fatcat:uuocxl66cbhvtc7kkys3hpbgbu

Which *BERT? A Survey Organizing Contextualized Encoders [article]

Patrick Xia, Shijie Wu, Benjamin Van Durme
2020 arXiv   pre-print
Through this organization, we highlight important considerations when interpreting recent contributions and choosing which model to use.  ...  We present a survey on language representation learning with the aim of consolidating a series of shared lessons learned across a variety of recent efforts.  ...  In addition, we thank Sabrina Mielke, Nathaniel Weir, Huda Khayrallah, Mitchell Gordon, and Shuoyang Ding for discussing several drafts of this work.  ... 
arXiv:2010.00854v1 fatcat:h4yxicrct5ccxb5zajtneuofrm

Joint Turn and Dialogue level User Satisfaction Estimation on Multi-Domain Conversations [article]

Praveen Kumar Bodigutla, Aditya Tiwari, Josep Valls Vargas, Lazaros Polymenakos, Spyros Matsoukas
2020 arXiv   pre-print
Current automated methods to estimate turn and dialogue level user satisfaction employ hand-crafted features and rely on complex annotation schemes, which reduce the generalizability of the trained models  ...  The proposed BiLSTM based deep neural net model automatically weighs each turn's contribution towards the estimated dialogue-level rating, implicitly encodes temporal dependencies, and removes the need  ...  Acknowledgments We thank Arindam Mandal and Jean-Jacques Loesch for their guidance and support. We thank Alexa Data Services-RAMP team for data. We also thank EMNLP reviewers for their feedback.  ... 
arXiv:2010.02495v2 fatcat:dslxpjilyrbjdbtdrrxhpyj674
« Previous Showing results 1 — 15 out of 171 results