A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
Filters
Linguistic Knowledge and Transferability of Contextual Representations
[article]
2019
arXiv
pre-print
Preserved Fulltext
Other Versions
To shed light on the linguistic knowledge they capture, we study the representations produced by several recent pretrained contextualizers (variants of ELMo, the OpenAI transformer language model, and ...
Contextual word representations derived from large-scale neural language models are successful across a diverse set of NLP tasks, suggesting that they encode useful and transferable features of language ...
We also thank the members of the Noah's ARK group at the University of Washington, the researchers at the Allen Institute for Artificial Intelligence, and the anonymous reviewers for their valuable feedback ...
arXiv:1903.08855v5
fatcat:74y4i5pf3resnj34mlkl2jrium
Linguistic Knowledge and Transferability of Contextual Representations
2019
Proceedings of the 2019 Conference of the North
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
To shed light on the linguistic knowledge they capture, we study the representations produced by several recent pretrained contextualizers (variants of ELMo, the OpenAI transformer language model, and ...
Contextual word representations derived from large-scale neural language models are successful across a diverse set of NLP tasks, suggesting that they encode useful and transferable features of language ...
We also thank the members of the Noah's ARK group at the University of Washington, the researchers at the Allen Institute for Artificial Intelligence, and the anonymous reviewers for their valuable feedback ...
doi:10.18653/v1/n19-1112
dblp:conf/naacl/Liu0BPS19
fatcat:uxpjqb7v3vgtjpxyz3ansjwubm
Improving CTC-based speech recognition via knowledge transferring from pre-trained language models
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
To solve this issue, we propose two knowledge transferring methods that leverage pre-trained LMs, such as BERT and GPT2, to improve CTC-based models. ...
When compared to the vanilla CTC-based models fine-tuned from the wav2vec2.0 models, our knowledge transferring method reduces CER by 16.1% relatively without external LMs. ...
contextual knowledge of pre-trained LMs into the ASR systems and we propose two kinds of knowledge transferring methods. ...
arXiv:2203.03582v1
fatcat:tyra453gkjfethxt6ysaomi2fi
Six Steps from Visual Metaphors to Knowledge Transfer
2015
Mediterranean Journal of Social Sciences
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL.
The file type is application/pdf.
The paper presents evidence from research into the problem of knowledge transfer modalities and cognitive powers of the visual metaphor. ...
The Procedure has been applied for the extraction of knowledge, transferred via visual metaphors in the World War II propaganda posters by KUKRYNIKSI. ...
which the projects on the study of knowledge transfer and conceptual modelling were completed. ...
doi:10.5901/mjss.2015.v6n6s5p228
fatcat:uyyyy7hm3ze5rbtaq2q7tntu64
Inferring symmetry in natural language
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Our results show that a hybrid transfer learning model that integrates linguistic features with contextualized language models most faithfully predicts the empirical data. ...
We develop methods that formalize these approaches and evaluate them against a novel symmetry inference sentence (SIS) dataset comprised of 400 naturalistic usages of literature-informed verbs spanning ...
YX is funded through a Connaught New Researcher Award, a NSERC Discovery Grant RGPIN-2018-05872, and a SSHRC Insight Grant #435190272. ...
arXiv:2010.08090v1
fatcat:bt7jbsyp4nhvvizqysjnzohvkm
Metaphors in Pre-Trained Language Models: Probing and Generalization Across Datasets and Languages
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Our extensive experiments suggest that contextual representations in PLMs do encode metaphorical knowledge, and mostly in their middle layers. ...
The knowledge is transferable between languages and datasets, especially when the annotation is consistent across training and testing sets. ...
Acknowledgements We would like to thank the anonymous reviewers and action editors who helped us greatly in improving our work with their comments. ...
arXiv:2203.14139v1
fatcat:tsrjljkzbvds3awwpms7msty4q
Cross-lingual Structure Transfer for Relation and Event Extraction
2019
Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
We exploit relation-and event-relevant language-universal features, leveraging both symbolic (including part-of-speech and dependency path) and distributional (including type representation and contextualized ...
We thus find that language-universal symbolic and distributional representations are complementary for cross-lingual structure transfer. ...
Linguistic knowledge and transferability of contextual representations. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics. ...
doi:10.18653/v1/d19-1030
dblp:conf/emnlp/SubburathinamLJ19
fatcat:3vwt72efyrfn7ojd6pkn64q7w4
Translation-Based Implicit Annotation Projection for Zero-Shot Cross-Lingual Event Argument Extraction
2022
Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
With the use of translation-based parallel corpora, no additional linguistic features are required during training and inference. ...
Moreover, our implicit annotation projection approach introduces less noises and hence is more effective and robust than explicit ones. ...
Science and Technology Program JSGG20210802154400001, and the Joint Lab of HITSZ and China Merchants Securities. ...
doi:10.1145/3477495.3531808
fatcat:73jwfzmwz5fihcvnuiv4ln3efm
Cross-Lingual Alignment of Contextual Word Embeddings, with Applications to Zero-shot Dependency Parsing
2019
Proceedings of the 2019 Conference of the North
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
While contextual embeddings have been shown to yield richer representations of meaning compared to their static counterparts, aligning them poses a challenge due to their dynamic nature. ...
We introduce a novel method for multilingual transfer that utilizes deep contextual embeddings, pretrained in an unsupervised fashion. ...
The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied, of ODNI, IARPA, or the ...
doi:10.18653/v1/n19-1162
dblp:conf/naacl/SchusterRBG19
fatcat:ns2bxzatkjdovnyqzxegtw53i4
Effect of Post-processing on Contextualized Word Representations
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
On a diverse set of pre-trained models, we show that post-processing unwraps vital information present in the representations for both lexical tasks (such as word similarity and analogy)and sequence classification ...
Our findings raise interesting points in relation to theresearch studies that use contextualized representations, and suggest z-score normalization as an essential step to consider when using them in an ...
Lin-
guistic knowledge and transferability of contextual
representations. ...
arXiv:2104.07456v1
fatcat:zc7ngcfy7nhcpchcj7dgw5kkxu
Pre-trained Models for Natural Language Processing: A Survey
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Next, we describe how to adapt the knowledge of PTMs to the downstream tasks. Finally, we outline some potential directions of PTMs for future research. ...
We first briefly introduce language representation learning and its research progress. Then we systematically categorize existing PTMs based on a taxonomy with four perspectives. ...
This work was supported by the National Natural Science Foundation of China (No. 61751201 and 61672162), Shanghai Municipal Science and Technology Major Project (No. 2018SHZDZX01) and ZJLab. ...
arXiv:2003.08271v3
fatcat:ze64wcfecfgs7bguq4vajpsgpu
Quantifying the Contextualization of Word Representations with Semantic Class Probing
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
for semantic class inference; and that top layer representations are more task-specific after finetuning while lower layer representations are more transferable. ...
We investigate the contextualization of words in BERT. ...
Acknowledgments We thank the anonymous reviewers for the insightful comments and suggestions. ...
arXiv:2004.12198v2
fatcat:crnslqh4jjb5bhizczlpdmtgay
On the Hierarchical Information in a Single Contextualised Word Representation (Student Abstract)
2020
PROCEEDINGS OF THE THIRTIETH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE AND THE TWENTY-EIGHTH INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
linguistic features encoded in their representations. ...
This work aims to investigate to what extent any linguistic hierarchical information is encoded into a single contextual embedding. ...
However, due to a poor understanding of the linguistic information contained in contextual representations -a thriving new area of research looks to employ novel analysis techniques to improve the interpretability ...
doi:10.1609/aaai.v34i10.7231
fatcat:itjoahgt5zgzdk7hnfcsshctuq
Learning Robust, Transferable Sentence Representations for Text Classification
[article]
2018
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Extensive experiments and analyses using a wide range of transfer and linguistic tasks endorse the effectiveness of our approach. ...
sentence representations that are useful for transfer learning. ...
The unified sentence encoder that combines both the sentence and contextual word representations captures most of the linguistic properties. Impact of training data. ...
arXiv:1810.00681v1
fatcat:6pkwhuzjrrh7bklpd7lxunf3nq
Translation by Quasi Logical Form transfer
1991
Proceedings of the 29th annual meeting on Association for Computational Linguistics -
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2004; you can also visit the original URL.
The file type is application/pdf.
Transfer takes place at the level of Quasi Logical Form (QLF), a contextually sensitive logical form representation which is deep enough for dealing with cross-linguistic differences. ...
Theoretical arguments and experimental results are presented to support the claim that this framework has good properties in terms of modularity, compositionality, reversibility and monotonicity. ...
ACKNOWLEDGMENTS The work reported here was funded by the Swedish Institute of Computer Science, and the greater part of it was carried out while the third author was employed there. ...
doi:10.3115/981344.981365
dblp:conf/acl/AlshawiCR91
fatcat:b73pt7ynbjbvtou6bfqgcci3eu
« Previous
Showing results 1 — 15 out of 42,685 results