A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Filters
Towards Incremental Transformers: An Empirical Analysis of Transformer Models for Incremental NLU
[article]
2021
arXiv
pre-print
Preserved Fulltext
In this work, we examine the feasibility of LT for incremental NLU in English. ...
Our results show that the recurrent LT model has better incremental performance and faster inference speed compared to the standard Transformer and LT with restart-incrementality, at the cost of part of ...
Acknowledgements We thank the anonymous reviewers for their critical reading of our manuscript and their insightful comments and suggestions. ...
arXiv:2109.07364v1
fatcat:3xvfbw5qa5bunnefcfp2okuu6u
Towards Incremental Transformers: An Empirical Analysis of Transformer Models for Incremental NLU
2021
Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
unpublished
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
In this work, we examine the feasibility of LT for incremental NLU in English. ...
Recent work attempts to apply Transformers incrementally via restart-incrementality by repeatedly feeding, to an unchanged model, increasingly longer input prefixes to produce partial outputs. ...
Acknowledgements We thank the anonymous reviewers for their critical reading of our manuscript and their insightful comments and suggestions. ...
doi:10.18653/v1/2021.emnlp-main.90
fatcat:npzzuy6z7bdgnn4gtwddclk6pa
IsoBN: Fine-Tuning BERT with Isotropic Batch Normalization
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Note that this fulltext copy is not of the "primary" version of this work. The version it corresponds to is:
2020 pre-print arXiv:2005.02178v1 fatcat:scnxjoh6yzdtjp4m7ly7diq4om
This simple yet effective fine-tuning method yields about 1.0 absolute increment on the average of seven NLU tasks. ...
Fine-tuning pre-trained language models (PTLMs), such as BERT and its better variant RoBERTa, has been a common practice for advancing performance in natural language understanding (NLU) tasks. ...
towards more isotropic representations, yielding an absolute increment around 1.0 point on 7 popular NLU tasks. ...
arXiv:2005.02178v2
fatcat:qrapp4bnzraythbjlollhpoef4
Graph-to-Graph Transformer for Transition-based Dependency Parsing
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
After proposing two novel Transformer models of transition-based dependency parsing as strong baselines, we show that adding the proposed mechanisms for conditioning on and predicting graphs of Graph2Graph ...
We propose the Graph2Graph Transformer architecture for conditioning on and predicting arbitrary graphs, and apply it to the challenging task of transition-based dependency parsing. ...
We also thank Lesly Miculicich, other members of the IDIAP NLU group, and anonymous reviewers for helpful discussions. ...
arXiv:1911.03561v4
fatcat:gn3k24ydhvbdrmqg77lu6gtxky
ClarET: Pre-training a Correlation-Aware Context-To-Event Transformer for Event-Centric Generation and Classification
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Other Versions
In this paper, we propose to pre-train a general Correlation-aware context-to-Event Transformer (ClarET) for event-centric reasoning. ...
The proposed ClarET is applicable to a wide range of event-centric reasoning scenarios, considering its versatility of (i) event-correlation types (e.g., causal, temporal, contrast), (ii) application formulations ...
Natural Language Understanding (NLU). Our basic model, BART-large, is presented for general NLU tasks. ...
arXiv:2203.02225v2
fatcat:dt4kivmupngujhwgv53e3p2lci
Current Challenges in Spoken Dialogue Systems and Why They Are Critical for Those Living with Dementia
[article]
2019
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
In this paper, we outline some of the challenges that are in urgent need of further research, including Incremental Speech Recognition and a systematic study of the interactional patterns in conversation ...
Dialogue technologies such as Amazon's Alexa have the potential to transform the healthcare industry. ...
A systematic, empirical study of such patterns is essential for informing design of dialogue systems for this target group -see Sec. 3.2. ...
arXiv:1909.06644v1
fatcat:ewmuxf7mhnasrmp4x2cihnvmny
Incremental Processing in the Age of Non-Incremental Encoders: An Empirical Assessment of Bidirectional Models for Incremental NLU
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
We test five models on various NLU datasets and compare their performance using three incremental evaluation metrics. ...
The results support the possibility of using bidirectional encoders in incremental mode while retaining most of their non-incremental quality. ...
Acknowledgments We are thankful to the three anonymous reviewers and to our colleagues in the Foundations of Computational Linguistics Lab at the University of Potsdam for their valuable comments and insights ...
arXiv:2010.05330v1
fatcat:wgspmk6ggze5voselhkqs26gca
ISIS
2004
ACM Transactions on Computer-Human Interaction
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL.
The file type is application/pdf.
ISIS provides a system test-bed for our work in multilingual speech recognition and generation, speaker authentication, language understanding and dialog modeling. ...
ISIS (Intelligent Speech for Information Systems) is a trilingual spoken dialog system (SDS) for the stocks domain. ...
We thank the past and present members of the ISIS team from the Human-Computer Communications Laboratory and Digital Signal Processing Laboratory of CUHK, as well as the National Key Laboratory for Machine ...
doi:10.1145/1017494.1017497
fatcat:hdiorq5t7nhlxgjw3um2oydcla
Non-Local Part-Aware Point Cloud Denoising
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
To enhance the denoising performance, we cascade a series of NLUs to progressively distill the noise features from the noisy inputs. ...
Further, besides the conventional surface reconstruction loss, we formulate a semantic part loss to regularize the predictions towards the relevant parts and enable denoising in a part-aware manner. ...
an incremental ablation study. ...
arXiv:2003.06631v1
fatcat:o7gjmwbdxndj5btadq37ite6uu
Sequence Package Analysis and Soft Computing: Introducing a New Hybrid Method to Adjust to the Fluid and Dynamic Nature of Human Speech
[chapter]
2011
Advances in Intelligent and Soft Computing
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2016; you can also visit the original URL.
The file type is application/pdf.
may transform a simple question into a rhetorical one; or transform an otherwise simple and straightforward assessment into a gratuitous/sardonic remark) cannot be adequately addressed by conventional ...
natural language speech interface, neural networks, or connectionist models, may be viewed as the natural choice for investigating the patterns underlying the orderliness of talk, as they are equipped ...
Perhaps this is the reason for the sea change in attitude toward acceptance of soft computing methods? ...
doi:10.1007/978-3-642-19644-7_1
fatcat:fehnwumlyja65fr3gtweb7773e
Detecting Egregious Conversations between Customers and Virtual Agents
[article]
2018
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
In this paper, we outline an approach to detecting such egregious conversations, using behavioral cues from the user, patterns in agent responses, and user-agent interaction. ...
Using logs of two commercial systems, we show that using these features improves the detection F1-score by around 20% over using textual features alone. ...
Feature Set Contribution Analysis To better understand the contributions of different sets of features to our EGR model, we examined various features in an incremental fashion. ...
arXiv:1711.05780v2
fatcat:5kzker2bqjckzjdi3wkklakjua
Detecting Egregious Conversations between Customers and Virtual Agents
2018
Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
In this paper, we outline an approach to detecting such egregious conversations, using behavioral cues from the user, patterns in agent responses, and useragent interaction. ...
Using logs of two commercial systems, we show that using these features improves the detection F1-score by around 20% over using textual features alone. ...
Feature Set Contribution Analysis To better understand the contributions of different sets of features to our EGR model, we examined various features in an incremental fashion. ...
doi:10.18653/v1/n18-1163
dblp:conf/naacl/SandbankSHKRP18
fatcat:vuppzshnxbedtffscd3y5kjke4
ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
In addition, most large-scale models are trained in an auto-regressive way. ...
Empirical results show that the model outperforms the state-of-the-art models on 54 Chinese NLP tasks, and its English version achieves the first place on the SuperGLUE benchmark (July 3, 2021), surpassing ...
Transformer-XL is similar to Transformer but introduces an auxiliary recurrence memory module to help modelling longer texts. ...
arXiv:2107.02137v1
fatcat:uuocxl66cbhvtc7kkys3hpbgbu
Which *BERT? A Survey Organizing Contextualized Encoders
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Through this organization, we highlight important considerations when interpreting recent contributions and choosing which model to use. ...
We present a survey on language representation learning with the aim of consolidating a series of shared lessons learned across a variety of recent efforts. ...
In addition, we thank Sabrina Mielke, Nathaniel Weir, Huda Khayrallah, Mitchell Gordon, and Shuoyang Ding for discussing several drafts of this work. ...
arXiv:2010.00854v1
fatcat:h4yxicrct5ccxb5zajtneuofrm
Joint Turn and Dialogue level User Satisfaction Estimation on Multi-Domain Conversations
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Current automated methods to estimate turn and dialogue level user satisfaction employ hand-crafted features and rely on complex annotation schemes, which reduce the generalizability of the trained models ...
The proposed BiLSTM based deep neural net model automatically weighs each turn's contribution towards the estimated dialogue-level rating, implicitly encodes temporal dependencies, and removes the need ...
Acknowledgments We thank Arindam Mandal and Jean-Jacques Loesch for their guidance and support. We thank Alexa Data Services-RAMP team for data. We also thank EMNLP reviewers for their feedback. ...
arXiv:2010.02495v2
fatcat:dslxpjilyrbjdbtdrrxhpyj674
« Previous
Showing results 1 — 15 out of 171 results