Filters








259 Hits in 2.7 sec

Word2vec Skip-gram Dimensionality Selection via Sequential Normalized Maximum Likelihood [article]

Pham Thuc Hung, Kenji Yamanishi
2020 arXiv   pre-print
In this paper, we propose a novel information criteria-based approach to select the dimensionality of the word2vec Skip-gram (SG).  ...  We examine the following information criteria for the dimensionality selection problem: the Akaike Information Criterion, Bayesian Information Criterion, and Sequential Normalized Maximum Likelihood (SNML  ...  We specifically propose the Sequential Normalized Maximum Likelihood (SNML) criterion in combination with some heuristics for the dimensionality selection problem.  ... 
arXiv:2008.07720v3 fatcat:iok5x5p65rd5jmljedwoxwh5ju

Word2vec Skip-Gram Dimensionality Selection via Sequential Normalized Maximum Likelihood

Pham Thuc Hung, Kenji Yamanishi
2021 Entropy  
In this paper, we propose a novel information criteria-based approach to select the dimensionality of the word2vec Skip-gram (SG).  ...  We examine the following information criteria for the dimensionality selection problem: the Akaike's Information Criterion (AIC), Bayesian Information Criterion (BIC), and Sequential Normalized Maximum  ...  ., Skip-gram model).  ... 
doi:10.3390/e23080997 fatcat:hb6l2dn5uzhq3hx2buj6n6zvnm

Neural Information Retrieval: A Literature Review [article]

Ye Zhang, Md Mustafizur Rahman, Alex Braylan, Brandon Dang, Heng-Lu Chang, Henna Kim, Quinten McNamara, Aaron Angert, Edward Banner, Vivek Khetan, Tyler McDonnell, An Thanh Nguyen (+3 others)
2017 arXiv   pre-print
Selection among word2vec CBOW or skip-gram or GloVe appears quite varied. Zuccon et al. (2015) compare CBOW vs. skip-gram, finding "no statistical significant differences between the two..."  ...  The authors induce word embeddings via word2vec CBOW only, though note that skip-gram embeddings could be used interchangeably.  ... 
arXiv:1611.06792v3 fatcat:i2eqfj5l25epjcytgvifta4y4i

Getting Started with Neural Models for Semantic Matching in Web Search [article]

Kezban Dilek Onal, Ismail Sengor Altingovde, Pinar Karagoz, Maarten de Rijke
2016 arXiv   pre-print
Neural language model The majority of the studies rely on CBOW and Skip-Gram models from the Word2Vec framework. Surprisingly, the Glove and Word2Vec models are not compared in any of the studies.  ...  In these models, low-dimensional word vectors are obtained via factorisation of a highdimensional sparse co-occurrence matrix.  ... 
arXiv:1611.03305v1 fatcat:agdgj7allbczxcyteuomswn574

Neural information retrieval: at the end of the early years

Kezban Dilek Onal, Ye Zhang, Ismail Sengor Altingovde, Md Mustafizur Rahman, Pinar Karagoz, Alex Braylan, Brandon Dang, Heng-Lu Chang, Henna Kim, Quinten McNamara, Aaron Angert, Edward Banner (+7 others)
2017 Information retrieval (Boston)  
Choice of NLM Selection among word2vec CBOW or Skip-gram or GloVe appears quite varied.  ...  The authors induce word embeddings via word2vec CBOW only, though they note that Skip-gram embeddings could be used interchangeably.  ... 
doi:10.1007/s10791-017-9321-y fatcat:plrhhwkppjgb7l5r5daiyryj4q

Survey of Neural Text Representation Models

Karlo Babić, Sanda Martinčić-Ipšić, Ana Meštrović
2020 Information  
Similarly to Word2Vec, it has two separate models, CBOW and skip-gram. It learns representations for character n-grams, and each word is represented as a bag-of-character n-grams.  ...  Word2Vec [22, 47] popularized shallow word representation using neural networks. Word2Vec has two separate autoencoding models: continuous bag-of-words (CBOW) and skip-gram.  ... 
doi:10.3390/info11110511 fatcat:veamykmme5cm5jhsllyc4xl7ma

Mixed Membership Word Embeddings for Computational Social Science [article]

James Foulds
2018 arXiv   pre-print
The experimental results show an improvement in predictive language modeling of up to 63% in MRR over the skip-gram, and demonstrate that the representations are beneficial for supervised learning.  ...  The MMSG can in principle be trained via maximum likelihood estimation using EM.  ...  With sufficiently high dimensional embeddings, the log-bilinear model can capture any distribution p(w c |z i ), and so the maximum likelihood embeddings would encode the exact same word distributions  ... 
arXiv:1705.07368v3 fatcat:47dyqfvxvvhqderbk2atx7bfpy

Skip-gram-KR: Korean Word Embedding for Semantic Clustering

Sun-Young Ihm, Ji-Hye Lee, Young-Ho Park
2019 IEEE Access  
In this paper, we propose a word embedding method for Korean, which is called Skip-gram-KR, and a Korean affix tokenizer.  ...  Skip-gram-KR creates similar word training data through backward mapping and the two-word skipping method. The experiment results show the proposed method achieved the most accurate performance.  ...  It is a word extraction method for Skip-gram-KR, and it is also a method which preserves the sequential meaning of words.  ... 
doi:10.1109/access.2019.2905252 fatcat:uw25dj3hljgerp6dwcasojev6u

Text-based depression detection on sparse data [article]

Heinrich Dinkel, Mengyue Wu, Kai Yu
2020 arXiv   pre-print
word embeddings like Word2Vec [20] which consists of the Continuous Bag of Words (CBoW) and the Skip-gram models, fastText [21] , [22] , as well as GloVe [23] .  ...  The penultimate BERT model layer was used to extract a single 768 dimensional sentence embedding. A maximum sequence length of 125 was set in order to reduce memory consumption.  ... 
arXiv:1904.05154v3 fatcat:2ly3v7avpreozhmiys3ffwzywy

Improving Neural Sequence Labelling Using Additional Linguistic Information

Mahtab Ahmed, Muhammad Rifayat Samee, Robert Mercer
2018 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA)  
These Deep Learning based models are outperforming traditional machine learning techniques by using abstract high dimensional feature representations of the input data.  ...  I would like to thank my parents for believing in me and for providing me with continuous support and encouragement throughout my graduate study. i Abstract Sequence Labelling is the task of mapping sequential  ...  We train this CRF model using maximum likelihood estimation (MLE) [18] .  ... 
doi:10.1109/icmla.2018.00104 dblp:conf/icmla/AhmedSM18 fatcat:m4xfmityorawjbpu34v6o2drhe

Application of an emotional classification model in e-commerce text based on an improved transformer model

Xuyang Wang, Yixuan Tong
2021 PLoS ONE  
This paper selects e-commerce reviews as research objects and applies deep learning theory. First, the text is preprocessed by word vectorization.  ...  Subsequently, this paper utilizes batchsize, sequencelen and embeddingsize to construct a 3-dimensional matrix. The skip-gram model is displayed in Fig 6.  ...  Using the skip-gram model in word2vec, the vector dimension is set to 100, the number of iterations is 8 and the training result is saved in the format of bin.  ... 
doi:10.1371/journal.pone.0247984 pmid:33667262 fatcat:3ne2alvlure2dndgexqkrvl24q

Customer Loyalty Improves the Effectiveness of Recommender Systems Based on Complex Network

Yun Bai, Suling Jia, Shuangzhe Wang, Binkai Tan
2020 Information  
The Word2Vec model has two training methods, continuous bag-of-word (CBOW) and skip-gram.  ...  The Word2Vec model has two training methods, continuous bag-of-word (CBOW) and skip-gram.  ... 
doi:10.3390/info11030171 fatcat:63wbxcpddvh7xeqlx5tbka6dz4

Vector representations of text data in deep learning [article]

Karol Grzegorczyk
2019 arXiv   pre-print
In addition to quantitative results, we present qualitative evaluation of Disambiguated Skip-gram, including two-dimensional visualisations of selected word-sense embeddings.  ...  For word-level representations we propose Disambiguated Skip-gram: a neural network model for learning multi-sense word embeddings.  ...  In addition to quantitative results, we present qualitative evaluation of Disambiguated Skip-gram, including two-dimensional visualisations of selected word-sense embeddings.  ... 
arXiv:1901.01695v1 fatcat:et6cxs45mbcipfyvblntjwpuge

Multi-dimension Topic Mining Based on Hierarchical Semantic Graph Model

Tingting Zhang, Baozhen Lee, Qinghua Zhu, Xi Han, Edwin Mouda Ye
2020 IEEE Access  
In addition, the multi-dimensional features of the topic can be mined effectively via an in-depth analysis of the constructed graph, resulting in a quantitative visualization of the many-to-many association  ...  INDEX TERMS Topic mining, multi-dimensional topic, hierarchical semantic graph.  ...  Word2vec can utilize either of two model architectures to produce a distributed representation of words: continuous bag-of-words (CBOW) or continuous skip-gram (CSG) [48] .  ... 
doi:10.1109/access.2020.2984352 fatcat:v6cu553qe5hg7j7gcihhmuh6ku

Deep Text Mining of Instagram Data without Strong Supervision

Kim Hammar, Shatha Jaradat, Nima Dokoohaki, Mihhail Matskin
2018 2018 IEEE/WIC/ACM International Conference on Web Intelligence (WI)  
By using the observed probabilities of overlap, the accuracy of each labeling function can be estimated using maximum likelihood estimation.  ...  State-of-the-art Algorithms Word2vec ) is a software package that implements both the Skip-gram model and the CBOW model.  ... 
doi:10.1109/wi.2018.00-94 dblp:conf/webi/HammarJDM18 fatcat:2x3qskb7njhcfhqf6rqcd7ti6y
« Previous Showing results 1 — 15 out of 259 results