Filters








360 Hits in 3.9 sec

A General and Multi-lingual Phrase Chunking Model Based on Masking Method [chapter]

Yu-Chieh Wu, Chia-Hui Chang, Yue-Shi Lee
2006 Lecture Notes in Computer Science  
In this paper, we propose a mask method to improve the chunking accuracy.  ...  The complete chunking time of a 50K words document is about 50 seconds.  ...  Acknowledgement This work is sponsored by National Science Council, Taiwan under grant NSC94-2524-S-008-002 and NSC94-2622-E-130-001-CC3.  ... 
doi:10.1007/11671299_17 fatcat:kbg352emune3bigqj6ewl4ssd4

Multilingual Code-Switching for Zero-Shot Cross-Lingual Intent Prediction and Slot Filling [article]

Jitin Krishnan, Antonios Anastasopoulos, Hemant Purohit, Huzefa Rangwala
2021 arXiv   pre-print
Experiments on the benchmark dataset of MultiATIS++ yielded an average improvement of +4.2% in accuracy for intent task and +1.8% in F1 for slot task using our method over the state-of-the-art across 8  ...  Our work focuses on a particular scenario where the target language is unknown during training.  ...  ., 2020) method is not a suitable baseline. Rather, we set this to be an upper bound, i.e. translating to the target language and fine-tuning the model should intuitively outperform a generic model.  ... 
arXiv:2103.07792v2 fatcat:t6pk5u3wfzbxrdjnfnlarwvtuy

MaskParse@Deskin at SemEval-2019 Task 1: Cross-lingual UCCA Semantic Parsing using Recursive Masked Sequence Tagging [article]

Gabriel Marzinotto, Geraldine Damnati
2019 arXiv   pre-print
Parsing is done recursively, we perform a first inference on the sentence to extract the main scenes and links and then we recursively apply our model on the sentence using a masking feature that reflects  ...  We choose a standard neural tagger and we focused on our recursive parsing strategy and on the cross lingual transfer problem to develop a robust model for the French language, using only few training  ...  (A) we do not allow to split multi word expressions (MWE) and chunks into different graph nodes.  ... 
arXiv:1910.02733v1 fatcat:vznvziicpfbqxlnqhbegqu3pdi

ERNIE: Enhanced Representation through Knowledge Integration [article]

Yu Sun, Shuohuan Wang, Yukun Li, Shikun Feng, Xuyi Chen, Han Zhang, Xin Tian, Danxiang Zhu, Hao Tian, Hua Wu
2019 arXiv   pre-print
Inspired by the masking strategy of BERT, ERNIE is designed to learn language representation enhanced by knowledge masking strategies, which includes entity-level masking and phrase-level masking.  ...  We also demonstrate that ERNIE has more powerful knowledge inference capacity on a cloze test.  ...  Based on basic level mask, we can obtain a basic word representation. Because it is trained on a random mask of basic semantic units, high level semantic knowledge is hard to be fully modeled.  ... 
arXiv:1904.09223v1 fatcat:tgbhnpobindobkzv5zwpnw7kg4

MaskParse@Deskin at SemEval-2019 Task 1: Cross-lingual UCCA Semantic Parsing using Recursive Masked Sequence Tagging

Gabriel Marzinotto, Johannes Heinecke, Géraldine Damnati
2019 Proceedings of the 13th International Workshop on Semantic Evaluation  
Parsing is done recursively, we perform a first inference on the sentence to extract the main scenes and links and then we recursively apply our model on the sentence using a masking feature that reflects  ...  We choose a standard neural tagger and we focused on our recursive parsing strategy and on the cross lingual transfer problem to develop a robust model for the French language, using only few training  ...  (A) we do not allow to split multi word expressions (MWE) and chunks into different graph nodes.  ... 
doi:10.18653/v1/s19-2015 dblp:conf/semeval/MarzinottoHD19 fatcat:kcwhierdu5erfhlzur5qnimtja

Linguistically Driven Multi-Task Pre-Training for Low-Resource Neural Machine Translation

Zhuoyuan Mao, Chenhui Chu, Sadao Kurohashi
2022 ACM Transactions on Asian and Low-Resource Language Information Processing  
JASS focuses on masking and reordering Japanese linguistic units known as bunsetsu, whereas ENSS is proposed based on phrase structure masking and reordering tasks.  ...  Adequacy evaluation using LASER, human evaluation, and case studies reveals that our proposed methods significantly outperform pre-training methods without injected linguistic knowledge and they have a  ...  One is phrase structure-based MASS (PMASS), and the other method is head finalization-based sequence-to-sequence pre-training (HFSS).  ... 
doi:10.1145/3491065 fatcat:plugwg6sxvfczlteqbgnbx5lpu

Benchmarking Multi-Task Learning for Sentiment Analysis and Offensive Language Identification in Under-Resourced Dravidian Languages [article]

Adeep Hande and Siddhanth U Hegde and Ruba Priyadharshini and Rahul Ponnusamy and Prasanna Kumar Kumaresan and Sajeetha Thavareesan and Bharathi Raja Chakravarthi
2021 arXiv   pre-print
Analysis of fine-tuned models indicates the preference of multi-task learning over single-task learning resulting in a higher weighted F1-score on all three languages.  ...  Experiments show that our multi-task learning model can achieve high results compared with single-task learning while reducing the time and space constraints required to train the models on individual  ...  In-dicBERT [102] is a fastText-based word embeddings and ALBERT-based LM trained on 11 Indian languages along with English and is a multi-lingual model similar to mBERT [99] .  ... 
arXiv:2108.03867v1 fatcat:fdavatswhrfanhdoracqixxuvy

XeroAlign: Zero-Shot Cross-lingual Transformer Alignment [article]

Milan Gritta, Ignacio Iacobacci
2021 arXiv   pre-print
XLM-RA's text classification accuracy exceeds that of XLM-R trained with labelled data and performs on par with state-of-the-art models on a cross-lingual adversarial paraphrasing task.  ...  We introduce XeroAlign, a simple method for task-specific alignment of cross-lingual pretrained transformers such as XLM-R.  ...  This is despite a strong average improvement of +4.1 on MTOP, +5.7 on MultiATIS++ and +5.2 on MTOD for the XLM-R-large model (greater for the XLM-RA-base model).  ... 
arXiv:2105.02472v2 fatcat:wnttyap5wbblncujqf3nquoeqi

Pretrained Language Models for Text Generation: A Survey [article]

Junyi Li, Tianyi Tang, Wayne Xin Zhao, Jian-Yun Nie, Ji-Rong Wen
2022 arXiv   pre-print
Text generation based on PLMs is viewed as a promising approach in both academia and industry. In this paper, we provide a survey on the utilization of PLMs in text generation.  ...  We also include a summary of various useful resources and typical text generation applications based on PLMs.  ...  It is difficult to apply English-based PLMs to solve multi-lingual text generation tasks (e.g., multi-lingual machine translation).  ... 
arXiv:2201.05273v4 fatcat:pnffabspsnbhvo44gbaorhxc3a

Transformer-Based Approaches for Legal Text Processing

Ha-Thanh Nguyen, Minh-Phuong Nguyen, Thi-Hai-Yen Vuong, Minh-Quan Bui, Minh-Chau Nguyen, Tran-Binh Dang, Vu Tran, Le-Minh Nguyen, Ken Satoh
2022 The Review of Socionetwork Strategies  
Although the paper focuses on technical reporting, the novelty of its approaches can also be an useful reference in automated legal document processing using Transformer-based models.  ...  In addition, we introduce to the community two pretrained models that take advantage of parallel translations in legal domain, NFSP and NMSP.  ...  The UA team built an information retrieval system based on methods such as BM25 (UA.BM25), TF-IDF(UA.tfidf) and language model (UA.LM). Method Task 1 and Task 2.  ... 
doi:10.1007/s12626-022-00102-2 fatcat:mii4xqiksbgtppvfeebmirk6pm

Multi-Task Learning in Natural Language Processing: An Overview [article]

Shijie Chen, Yu Zhang, Qiang Yang
2021 arXiv   pre-print
Then we present optimization techniques on loss construction, data sampling, and task scheduling to properly train a multi-task model.  ...  However, deep neural models often suffer from overfitting and data scarcity problems that are pervasive in NLP tasks.  ...  One task is the multi-lingual skip-gram task where a model predicts masked words according to both monolingual and cross-lingual contexts and another task is the cross-lingual sentence similarity estimation  ... 
arXiv:2109.09138v1 fatcat:hlgzjykuvzczzmsgnl32w5qo5q

Large-scale multilingual audio visual dubbing [article]

Yi Yang, Brendan Shillingford, Yannis Assael, Miaosen Wang, Wendi Liu, Yutian Chen, Yu Zhang, Eren Sezener, Luis C. Cobo, Misha Denil, Yusuf Aytar, Nando de Freitas
2020 arXiv   pre-print
The audio and visual translation subsystems each contain a large-scale generic synthesis model trained on thousands of hours of data in the corresponding domain.  ...  We describe a system for large-scale audiovisual translation and dubbing, which translates videos from one language to another.  ...  We would like to thank Martin Aguinis, Paige Bailey, and Laurence Moroney for their permission to use their Tensorflow tutorial videos to demonstrate our end-to-end video dubbing approach, and for allowing  ... 
arXiv:2011.03530v1 fatcat:xzgcifgigvakhetruzsseal36i

Learning to Represent Words in Context with Multilingual Supervision [article]

Kazuya Kawakami, Chris Dyer
2015 arXiv   pre-print
To learn the parameters of our model, we use cross-lingual supervision, hypothesizing that a good representation of a word in context will be one that is sufficient for selecting the correct translation  ...  We present a neural network architecture based on bidirectional LSTMs to compute representations of words in the sentential contexts.  ...  Furthermore, one can learn reasonable generalizations from models that condition on and the generate text using an autoencoding objective (Socher et al., 2011) .  ... 
arXiv:1511.04623v2 fatcat:mufop3p7rfc5td6oahe7s7y3yi

Actuarial Applications of Natural Language Processing Using Transformers: Case Studies for Using Text Features in an Actuarial Context [article]

Andreas Troxler
2022 arXiv   pre-print
The case studies tackle challenges related to a multi-lingual setting and long input sequences.  ...  This tutorial demonstrates workflows to incorporate text data into actuarial classification and regression tasks. The main focus is on methods employing transformer-based models.  ...  Acknowledgements The authors are very grateful to Mario Wüthrich, Christian Lorentzen and Michael Mayer for their comprehensive reviews and their innumerable inputs which led to substantial improvements  ... 
arXiv:2206.02014v1 fatcat:i2bqzfvfb5ho5kpfjgmfppxccu

JNLP Team: Deep Learning Approaches for Legal Processing Tasks in COLIEE 2021 [article]

Ha-Thanh Nguyen, Phuong Minh Nguyen, Thi-Hai-Yen Vuong, Quan Minh Bui, Chau Minh Nguyen, Binh Tran Dang, Vu Tran, Minh Le Nguyen, Ken Satoh
2021 arXiv   pre-print
In this article, we survey and report our methods and experimental results in using deep learning in legal document processing.  ...  Automatic legal document processing is an ambitious goal, and the structure and semantics of the law are often far more complex than everyday language.  ...  ACKNOWLEDGMENTS This work was supported by JSPS KAKENHI Grant Numbers JP17H06103 and JP20K20406.  ... 
arXiv:2106.13405v2 fatcat:5kplhbi3qnhyzogsmavrobsyvi
« Previous Showing results 1 — 15 out of 360 results