86 Hits in 3.3 sec

Robust Gram Embeddings

Taygun Kekec, David M. J. Tax
2016 Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing  
We propose a regularized embedding formulation, called Robust Gram (RG), which penalizes overfitting by suppressing the disparity between target and context embeddings.  ...  When training data is scarce, these models risk losing their generalization abilities due to the complexity of the models and the overfitting to finite data.  ...  Acknowledgments The authors acknowledge funding by the Dutch Organization for Scientific Research (NWO; grant 612.001.301).  ... 
doi:10.18653/v1/d16-1113 dblp:conf/emnlp/KekecT16 fatcat:nauo2v6g3zapdbvodwklnwjftq

Heterogeneous Ensemble Deep Learning Model for Enhanced Arabic Sentiment Analysis

Hager Saleh, Sherif Mostafa, Abdullah Alharbi, Shaker El-Sappagh, Tamim Alkhalifah
2022 Sensors  
Traditional machine learning and deep neural algorithms have been used in a variety of studies to predict sentiment analysis.  ...  meta-learners Logistic Regression (LR), Random Forest (RF), and Support Vector Machine (SVM) in order to enhance model's performance for predicting Arabic sentiment analysis.  ...  Acknowledgments: The researchers would like to thank the Deanship of Scientific Research, Qassim University for funding the publication of this project.  ... 
doi:10.3390/s22103707 pmid:35632116 pmcid:PMC9147256 fatcat:ssga3cqsanedxdqydb5el2tmoq

Polarity Detection of Dialectal Arabic using Deep Learning Models

Saleh M. Mohamed, Ensaf Hussein Mohamed, Mohamed A. Belal
2021 International Journal of Advanced Computer Science and Applications  
The performance of the four models is validated on the used corpus with the use of word embedding and applying the (k-Fold Cross-Validation) method.  ...  With the evolution of a new era of technology and social media networks, as well as an increase in Arabs sharing their point of view, it became necessary that this research be conducted.  ...  There have been a few studies that work on DA or mix both MSA and DA, specifically (MSA/Egyptian). Table I summarises these studies by author, year, and the used methodology.  ... 
doi:10.14569/ijacsa.2021.0121125 fatcat:vf2hajcutnda7mk2k4ce732edm

A Text Emotion Analysis Method Using the Dual-Channel Convolution Neural Network in Social Networks

Di Wu, Jianpei Zhang, Qingchao Zhao, Yi-Zhang Jiang
2020 Mathematical Problems in Engineering  
Finally, a dynamic k-max continuous pooling strategy is adopted to realize the dimensionality reduction of features and enhance the model's ability to extract features.  ...  Then, using the CNN's multichannel mechanism, the extended text features based on the word vector features and the semantic features based on the word vectors are, respectively, input into the CNN model  ...  on the CBOW training model; η 2 represents the weight of the word vector based on the SkipGram training model; and Z represents the memory size of the corpus size in GB.  ... 
doi:10.1155/2020/6182876 fatcat:au37d7yovvbjhn7qzbvykmgrae

Combining word embeddings and convolutional neural networks to detect duplicated questions [article]

Yoan Dimitrov
2020 arXiv   pre-print
We experiment with different embedding approaches such as Word2Vec, Fasttext, and Doc2Vec and investigate the effects these approaches have on model performance.  ...  Our model achieves competitive results on the Quora dataset and complements the well-established evidence that CNNs can be utilized for paraphrase detection tasks.  ...  To avoid overfitting and stabilize training, we apply a dropout rate [23] of 0.1 after every convolutional layer and use batch normalization [8] with a momentum of 0.7 after every embedding layer.  ... 
arXiv:2006.04513v1 fatcat:scvg5guuq5e6vcgvpk2lxyxetu

DNA sequences performs as natural language processing by exploiting deep learning algorithm for the identification of N4-methylcytosine

Abdul Wahab, Hilal Tayara, Zhenyu Xuan, Kil To Chong
2021 Scientific Reports  
In the proposed work, a computational model, 4mCNLP-Deep, used the word embedding approach as a vector formulation by exploiting deep learning based CNN algorithm to predict 4mC and non-4mC sites on the  ...  AbstractN4-methylcytosine is a biochemical alteration of DNA that affects the genetic operations without modifying the DNA nucleotides such as gene expression, genomic imprinting, chromosome stability,  ...  Figure 3 , demonstrates the visualization of the mutation on the C. elegans dataset as an indigenous feature during the model's learning phase.  ... 
doi:10.1038/s41598-020-80430-x pmid:33420191 fatcat:osfe627fszhnriqt7r5srddsmq

Low Frequency Names Exhibit Bias and Overfitting in Contextualizing Language Models [article]

Robert Wolfe, Aylin Caliskan
2021 arXiv   pre-print
We use a dataset of U.S. first names with labels based on predominant gender and racial group to examine the effect of training corpus frequency on tokenization, contextualization, similarity to initial  ...  Representations of infrequent names undergo more processing, but are more self-similar, indicating that models rely on less context-informed representations of uncommon and minority names which are overfit  ...  In the context of studies of race, Sen and Wasow (2016) usefully define an exposure study as one which uses "a cue or signal that generates some reaction," and note that "names often act as a proxy for  ... 
arXiv:2110.00672v1 fatcat:oqgtqpse45d3vaz7zfvalbdnee

Ensembling Classical Machine Learning and Deep Learning Approaches for Morbidity Identification from Clinical Notes

Vivek Kumar, Diego Reforgiato Recupero, Daniele Riboni, Rim Helaoui
2020 IEEE Access  
The goal of our work is to develop a classification system to identify whether a certain health condition occurs for a patient by studying his/her past clinical records.  ...  Analysis and interpretation of healthcare data is a daunting task, and it demands a great deal of time, resources, and human effort.  ...  Her role in the project is not a data controller, nor a data processor, but a knowledge contributor.  ... 
doi:10.1109/access.2020.3043221 fatcat:szn2nkczize4hlzleuzjfrnu5m

Japanese Mistakable Legal Term Correction using Infrequency-aware BERT Classifier

2020 Transactions of the Japanese society for artificial intelligence  
The former causes a class imbalance problem and the latter causes an underfitting problem; both degrade classification performance.  ...  Our method predicts suitable legal terms using a classifier based on BERT (Bidirectional Encoder Representations from Transformers).  ...  Some of the above studies focus on word-level correction (e.g., [Takeda 86, Chae 98]), which is the same as our scope.  ... 
doi:10.1527/tjsai.e-k25 fatcat:v3nt46jntvhtpdotad6cdej7q4

BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer [article]

Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, Peng Jiang
2019 arXiv   pre-print
To address this problem, we train the bidirectional model using the Cloze task, predicting the masked items in the sequence by jointly conditioning on their left and right context.  ...  However, jointly conditioning on both left and right context in deep bidirectional model would make the training become trivial since each item can indirectly "see the target item".  ...  From this point of view, Cloze is a general form for the objective of CBOW and SG.  ... 
arXiv:1904.06690v2 fatcat:vivnjneyvjcclaya4ep2slm62m

Research on Public Environmental Perception of Emotion, Taking Haze as an Example

Qiang Bao, Xujuan Zhang, Xijuan Wu, Qiang Zhang, Jinshou Chen
2021 International Journal of Environmental Research and Public Health  
Environmental problems represented by haze have become a topic that affects the harmonious ecology of human beings. The trend of this topic is on the rise.  ...  about fog and haze as environmental perception data, and analyzes the impact of fog and haze on the public in four seasonal time dimensions.  ...  Conflicts of Interest: The authors declare no conflict of interest.  ... 
doi:10.3390/ijerph182212115 pmid:34831869 pmcid:PMC8624140 fatcat:qkiusyvtmbem7ka2jbp3npooyy

A Comparative Analysis of Generative Neural Attention-based Service Chatbot

Sinarwati Mohamad Suhaili, Naomie Salim, Mohamad Nazim Jambli
2022 International Journal of Advanced Computer Science and Applications  
The primary focus is to automate the generation of conversational chat between a computer and a human by constructing virtual service agents that can predict appropriate and automatic responses to customers  ...  This paper aims to present and implement a seq2seq-based learning task model based on encoder-decoder architectural solutions by training generative chatbots on customer support Twitter datasets.  ...  The authors would also like to thank the Universiti Malaysia Sarawak (UNIMAS) and Universiti Teknologi Malaysia (UTM) for providing the resources used in this research work.  ... 
doi:10.14569/ijacsa.2022.0130885 fatcat:ppambo7hrrbjplnprkg535uwqm

Generalization in NLI: Ways (Not) To Go Beyond Simple Heuristics [article]

Prajjwal Bhargava, Aleksandr Drozd, Anna Rogers
2021 arXiv   pre-print
We conduct a case study of generalization in NLI (from MNLI to the adversarially constructed HANS dataset) in a range of BERT-based architectures (adapters, Siamese Transformers, HEX debiasing), as well  ...  as with subsampling the data and increasing the model size.  ...  The 'naive' model is a CBOW model with a self-attention layer (Vaswani et al., 2017) to capture co-occurrence information from the sequence with input and token embeddings.  ... 
arXiv:2110.01518v1 fatcat:orftxwh7uvcwdkpamabwqzxgti

The Long Arc of History: Neural Network Approaches to Diachronic Linguistic Change

Eun Seo Jo, Mark Algee-Hewitt
2018 Journal of the Japanese Association for Digital Humanities  
We model the language of our corpus as a whole in a way that is reminiscent of Juola's work but on a much bigger scale.  ...  Chauncey (1994) argues that the tense Cold War atmosphere and the end of the Prohibition changed the meaning of the term "gay" from being a descriptive quality of male identity based on gender (masculinity  ...  Our study of the series as a whole can inform scholars who base at least a part of their study on FRUS.  ... 
doi:10.17928/jjadh.3.1_1 fatcat:j27j6kj4jffbvmsmyyhkgoouhy

Boosting Convolutional Neural Networks Using a Bidirectional Fast Gated Recurrent Unit for Text Categorization

Assia Belherazem, Redouane Tlemsani
2022 International Journal of Artificial Intelligence and Machine Learning  
Precision/loss score was used as the main criterion on five different datasets (WebKb, R8, R52, AG-News, and 20 NG) to assess the performance of the proposed model.  ...  The results indicate that the precision score of the classifier on WebKb, R8, and R52 data sets significantly improved from 90% up to 97% compared to the best result achieved by other methods such as LSTM  ...  to the content of this article.  ... 
doi:10.4018/ijaiml.308815 fatcat:x7bdmojozfaixk4b6hmxmrramm
« Previous Showing results 1 — 15 out of 86 results