Filters








51,326 Hits in 7.3 sec

Word Representations: A Simple and General Method for Semi-Supervised Learning

Joseph P. Turian, Lev-Arie Ratinov, Yoshua Bengio
2010 Annual Meeting of the Association for Computational Linguistics  
If we take an existing supervised NLP system, a simple and general way to improve accuracy is to use unsupervised word representations as extra word features.  ...  We use near state-of-the-art supervised baselines, and find that each of the three word representations improves the accuracy of these baselines.  ...  Joseph Turian and Yoshua Bengio acknowledge the following agencies for research funding and computing support: NSERC, RQCHP, CIFAR.  ... 
dblp:conf/acl/TurianRB10 fatcat:2r32qohk5fcetif2mkffd5jrxy

Deep Contextualized Word Representations

Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, Luke Zettlemoyer
2018 Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)  
We introduce a new type of deep contextualized word representation that models both (1) complex characteristics of word use (e.g., syntax and semantics), and (2) how these uses vary across linguistic contexts  ...  Our word vectors are learned functions of the internal states of a deep bidirectional language model (biLM), which is pretrained on a large text corpus.  ...  types of semi-supervision.  ... 
doi:10.18653/v1/n18-1202 dblp:conf/naacl/PetersNIGCLZ18 fatcat:lbvmqppuo5cpllcubjt2ubb37m

Deep contextualized word representations [article]

Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, Luke Zettlemoyer
2018 arXiv   pre-print
We introduce a new type of deep contextualized word representation that models both (1) complex characteristics of word use (e.g., syntax and semantics), and (2) how these uses vary across linguistic contexts  ...  Our word vectors are learned functions of the internal states of a deep bidirectional language model (biLM), which is pre-trained on a large text corpus.  ...  Using biLMs for supervised NLP tasks Given a pre-trained biLM and a supervised architecture for a target NLP task, it is a simple process to use the biLM to improve the task model.  ... 
arXiv:1802.05365v2 fatcat:4fxzi2utynh25iqgx36lp5sila

Unsupervised Word Polarity Tagging by Exploiting Continuous Word Representations

Aitor García Pablos, Montse Cuadros, German Rigau
2015 Revista de Procesamiento de Lenguaje Natural (SEPLN)  
Palabras clave: word embeddings, polaridad de palabras, análisis de sentimiento  ...  Un lexicón de polaridad es un diccionario que asigna un valor predeteminado de polaridad a una palabra.  ...  There are many different approaches in the literature: some of them employ supervised machine learning methods to train a model that learns which words/expressions/sentences are positives and which are  ... 
dblp:journals/pdln/PablosCR15 fatcat:cqii7aymtjgt7gcrjlxofzi7ky

Inducing Crosslingual Distributed Representations of Words

Alexandre Klementiev, Ivan Titov, Binod Bhattarai
2012 International Conference on Computational Linguistics  
These representations can be used for a number of crosslingual learning tasks, where a learner can be trained on annotations present in one language and applied to test data in another.  ...  We treat it as a multitask learning problem where each task corresponds to a single word, and task relatedness is derived from co-occurrence statistics in bilingual parallel data.  ...  Acknowledgements The work was supported by the MMCI Cluster of Excellence and a Google research award.  ... 
dblp:conf/coling/KlementievTB12 fatcat:molecfn4t5apjd7uw2pw3xqg7y

Improved Word Sense Disambiguation Using Pre-Trained Contextualized Word Representations [article]

Christian Hadiwinoto, Hwee Tou Ng, Wee Chung Gan
2019 arXiv   pre-print
Contextualized word representations are able to give different representations for the same word in different contexts, and they have been shown to be effective in downstream natural language processing  ...  In this paper, we explore different strategies of integrating pre-trained contextualized word representations and our best strategy achieves accuracies exceeding the best prior published accuracies by  ...  Finally, we also compare with a prior multi-task and semi-supervised WSD approach learned through alternating structure optimization (ASO) (Ando, 2006) , which also utilizes unlabeled data for training  ... 
arXiv:1910.00194v2 fatcat:n755npphy5gibpsptkuzbhgtwy

How to Evaluate Word Representations of Informal Domain? [article]

Yekun Chai, Naomi Saphra, Adam Lopez
2019 arXiv   pre-print
We derived a large list of variant spelling pairs from UrbanDictionary with the automatic approaches of weakly-supervised pattern-based bootstrapping and self-training linear-chain conditional random field  ...  With these extracted relation pairs we promote the odds of eliding the text normalization procedure of traditional NLP pipelines and directly adopting representations of non-standard words in the informal  ...  Word representations: a simple and general method for semi-supervised learning. In Proceedings of the 48th annual meeting of the association for compu- tational linguistics, pages 384-394.  ... 
arXiv:1911.04669v2 fatcat:kvf3gptduvhhzfcximff6mimia

Arabic Named Entity Recognition using Word Representations [article]

Ismail El Bazi, Nabil Laachfoubi
2018 arXiv   pre-print
Recent work has shown the effectiveness of the word representations features in significantly improving supervised NER for the English language.  ...  We use word representations as additional features in a Conditional Random Field (CRF) model and we systematically compare three popular neural word embedding algorithms (SKIP-gram, CBOW and GloVe) and  ...  EXPERIMENTS AND DISCUSSION A. NER Model In this study, we follow a supervised machine learning approach.  ... 
arXiv:1804.05630v1 fatcat:ctioav6jcvghlldrklzrg3tmaq

Annotation-free Learning of Deep Representations for Word Spotting using Synthetic Data and Self Labeling [article]

Fabian Wolf, Gernot A. Fink
2020 arXiv   pre-print
Word spotting is a popular tool for supporting the first exploration of historic, handwritten document collections.  ...  In this work, we present an annotation-free method that still employs machine learning techniques and therefore outperforms other learning-free approaches.  ...  A special type of semi-supervised methods employ so called self-labeling techniques [29] , which have been also studied for neural networks [14] .  ... 
arXiv:2003.01989v4 fatcat:yagnjuknhree7g3mympqorq6qa

Massively Multilingual Sparse Word Representations

Gábor Berend
2020 International Conference on Learning Representations  
In this paper, we introduce MAMUS for constructing multilingual sparse word representations.  ...  Finally, we are releasing our multilingual sparse word representations for the 27 typologically diverse set of languages that we conducted our various experiments on.  ...  This research was supported by grant TUDFO/47138-1/2019-ITM of the Ministry for Innovation and Technology, Hungary.  ... 
dblp:conf/iclr/Berend20 fatcat:zoommhwkq5db7dqw6l4mk6v2ui

Improved Word Sense Disambiguation Using Pre-Trained Contextualized Word Representations

Christian Hadiwinoto, Hwee Tou Ng, Wee Chung Gan
2019 Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)  
Contextualized word representations are able to give different representations for the same word in different contexts, and they have been shown to be effective in downstream natural language processing  ...  In this paper, we explore different strategies of integrating pre-trained contextualized word representations and our best strategy achieves accuracies exceeding the best prior published accuracies by  ...  Finally, we also compare with a prior multi-task and semi-supervised WSD approach learned through alternating structure optimization (ASO) (Ando, 2006) , which also utilizes unlabeled data for training  ... 
doi:10.18653/v1/d19-1533 dblp:conf/emnlp/HadiwinotoNG19 fatcat:siwrvwgdsndnxo566qpr2rkbju

Integrating Approaches to Word Representation [article]

Yuval Pinter
2021 arXiv   pre-print
I present a survey of the distributional, compositional, and relational approaches to addressing this task, and discuss various means of integrating them into systems, with special emphasis on the word  ...  level and the out-of-vocabulary phenomenon.  ...  I thank my committee for helping to shape it: my advisor, Jacob Eisenstein; Mark Riedl, Dan Roth, Wei Xu, and Diyi Yang.  ... 
arXiv:2109.04876v1 fatcat:hyzt3j7ibrhe3dg72imc27w4zy

Learning Representations by Predicting Bags of Visual Words

Spyros Gidaris, Andrei Bursuc, Nikos Komodakis, Patrick Perez, Matthieu Cord
2020 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)  
Self-supervised representation learning targets to learn convnet-based image representations from unlabeled data.  ...  Then, as a self-supervised task, we train another convnet to predict the histogram of visual words of an image (i.e., its Bag-of-Words representation) given as input a perturbed version of that image.  ...  We would like to thank Gabriel de Marmiesse for his invaluable support during the experimental implementation and analysis of this work.  ... 
doi:10.1109/cvpr42600.2020.00696 dblp:conf/cvpr/GidarisBKPC20 fatcat:ypdi4xpwoffs3nyvqnkdyayzwe

Learning Representations by Predicting Bags of Visual Words [article]

Spyros Gidaris, Andrei Bursuc, Nikos Komodakis, Patrick Pérez, Matthieu Cord
2020 arXiv   pre-print
Self-supervised representation learning targets to learn convnet-based image representations from unlabeled data.  ...  Then, as a self-supervised task, we train another convnet to predict the histogram of visual words of an image (i.e., its Bag-of-Words representation) given as input a perturbed version of that image.  ...  We would like to thank Gabriel de Marmiesse for his invaluable support during the experimental implementation and analysis of this work.  ... 
arXiv:2002.12247v1 fatcat:rq622at6vfh2rcp76mpbwsubd4

Incorporating Both Distributional and Relational Semantics in Word Representations [article]

Daniel Fried, Kevin Duh
2015 arXiv   pre-print
To this end, we employ the Alternating Direction Method of Multipliers (ADMM), which flexibly optimizes a distributional objective on raw text and a relational objective on WordNet.  ...  We investigate the hypothesis that word representations ought to incorporate both distributional and relational semantics.  ...  Word representations: A simple and general method for semi-supervise learning.  ... 
arXiv:1412.5836v3 fatcat:rdkhaz3nznbohekjh2g7l47sv4
« Previous Showing results 1 — 15 out of 51,326 results