Filters








20,438 Hits in 3.1 sec

Language Modeling with Gated Convolutional Networks [article]

Yann N. Dauphin, Angela Fan, Michael Auli, David Grangier
2017 arXiv   pre-print
To our knowledge, this is the first time a non-recurrent approach is competitive with strong recurrent models on these large scale language tasks.  ...  The pre-dominant approach to language modeling to date is based on recurrent neural networks. Their success on this task is often linked to their ability to capture unbounded context.  ...  Conclusion We introduce a convolutional neural network for language modeling with a novel gating mechanism.  ... 
arXiv:1612.08083v3 fatcat:mnkgy43p7zgmxigcatcesj4yci

Advanced Convolutional Neural Network-Based Hybrid Acoustic Models for Low-Resource Speech Recognition

Tessfu Geteye Fantaye, Junqing Yu, Tulu Tilahun Hailu
2020 Computers  
This paper presents the results of our contributions to combine CNN and conventional RNN with gate, highway, and residual networks to reduce the above problems.  ...  Of these networks, convolutional neural network (CNN) is an effective network for representing the local properties of the speech formants.  ...  Gated Convolutional Neural Network Model The network configuration for training the GCNN model is given as follows: the input features of the GCNN model were the same as with the CNN model, which is a  ... 
doi:10.3390/computers9020036 fatcat:k54s5pj7grggffsbibjpm5jk2q

Sentiment Analysis in Myanmar Language using Convolutional LSTM Neural Network

Nwet Yin Tun Thein, Khin Mar Soe
2021 International Journal on Natural Language Computing  
In this paper, the Convolutional LSTM neural network architecture is proposed to analyse the sentiment classification of cosmetic reviews written in Myanmar Language.  ...  The paper also intends to build the cosmetic reviews dataset for deep learning and sentiment lexicon in Myanmar Language.  ...  NEURAL NETWORK ARCHITECTURE The proposed neural network architecture will be implemented by the combination of convolutional neural network with LSTM model.  ... 
doi:10.5121/ijnlc.2021.10402 fatcat:xyom7m7sqbcepeo4skhicnxseq

Multi-Language Identification Using Convolutional Recurrent Neural Network [article]

Vrishabh Ajay Lakhani, Rohan Mahadev
2017 arXiv   pre-print
To achieve this, we use the novel approach of using a Convolutional Recurrent Neural Network using Long Short Term Memory (LSTM) or a Gated Recurrent Unit (GRU) for forward propagation of the neural network  ...  Our hypothesis is that the performance of using polyphonic sound sequence as features and both LSTM and GRU as the gating mechanisms for the neural network outperform the traditional MFCC features using  ...  Traditionally, Convolutional recurrent neural network are used for scene labelling [7] , that is, using Convolutional Neural Networks with Intra-layer Recurrent Connections, we use this novel approach  ... 
arXiv:1611.04010v2 fatcat:glks4dpg75hfjjsj2ronijrkym

Encoding Source Language with Convolutional Neural Network for Machine Translation [article]

Fandong Meng and Zhengdong Lu and Mingxuan Wang and Hang Li and Wenbin Jiang and Qun Liu
2015 arXiv   pre-print
The recently proposed neural network joint model (NNJM) (Devlin et al., 2014) augments the n-gram target language model with a heuristically chosen source context window, achieving state-of-the-art performance  ...  This representation, together with target language words, are fed to a deep neural network (DNN) to form a stronger NNJM.  ...  Conclusion and Future Work We proposed convolutional architectures for obtaining a guided representation of the entire source sentence, which can be used to augment the n-gram target language model.  ... 
arXiv:1503.01838v5 fatcat:vdqhjqmh5zco3jf7em3em45cg4

genCNN: A Convolutional Architecture for Word Sequence Prediction [article]

Mingxuan Wang, Zhengdong Lu, Hang Li, Wenbin Jiang, Qun Liu
2015 arXiv   pre-print
Also different from the existing feedforward networks for language modeling, our model can effectively fuse the local correlation and global correlation in the word sequence, with a convolution-gating  ...  Instead, we use a convolutional neural network to predict the next word with the history of words of variable length.  ...  Conclusion We propose a convolutional architecture for natural language generation and modeling.  ... 
arXiv:1503.05034v2 fatcat:xhnmdoxoszfc7jl7lwl5wowqaa

genCNN: A Convolutional Architecture for Word Sequence Prediction

Mingxuan Wang, Zhengdong Lu, Hang Li, Wenbin Jiang, Qun Liu
2015 Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)  
Also different from the existing feedforward networks for language modeling, our model can effectively fuse the local correlation and global correlation in the word sequence, with a convolution-gating  ...  Instead, we use a convolutional neural network to predict the next word with the history of words of variable length.  ...  Conclusion We propose a convolutional architecture for natural language generation and modeling.  ... 
doi:10.3115/v1/p15-1151 dblp:conf/acl/WangLLJL15 fatcat:fvxljvdlofetvecchgkxwx72mu

Named Entity Recognition with Gated Convolutional Neural Networks [chapter]

Chunqi Wang, Wei Chen, Bo Xu
2017 Lecture Notes in Computer Science  
In this work, we propose a novel architecture for NER problems based on GCNN -CNN with gating mechanism.  ...  Recently, convolutional neural networks (CNNs) have achieved great success on computer vision. However, for NER problems, they are not well studied.  ...  We introduce gating mechanism into the convolutional layer. Dauphin [7] have shown that gating mechanism is useful for language modeling tasks.  ... 
doi:10.1007/978-3-319-69005-6_10 fatcat:qsszpa4inbcpbegbed3j7pffhm

Encoding Source Language with Convolutional Neural Network for Machine Translation

Fandong Meng, Zhengdong Lu, Mingxuan Wang, Hang Li, Wenbin Jiang, Qun Liu
2015 Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)  
The recently proposed neural network joint model (NNJM) (Devlin et al., 2014) augments the n-gram target language model with a heuristically chosen source context window, achieving state-of-the-art performance  ...  This representation, together with target language words, are fed to a deep neural network (DNN) to form a stronger NNJM.  ...  (before convolution) on Layer-0, and merge them for the input of the gating network.  ... 
doi:10.3115/v1/p15-1003 dblp:conf/acl/MengLWLJL15 fatcat:ien7dxizojdf7e72qrk7hriaou

Have Convolutions Already Made Recurrence Obsolete for Unconstrained Handwritten Text Recognition?

Denis Coquenet, Yann Soullard, Clement Chatelain, Thierry Paquet
2019 2019 International Conference on Document Analysis and Recognition Workshops (ICDARW)  
Recently, recurrence-free architectures such as Fully Convolutional Networks with gated mechanisms have been proposed as one possible alternative achieving competitive results.  ...  In this paper, we explore convolutional architectures and compare them to a CNN+BLSTM baseline.  ...  Indeed, the gates could compensate the selection work achieved by the LSTM cells. That is why we decided to compare a LSTM based model with a gated convolutional model (G-CNN). III.  ... 
doi:10.1109/icdarw.2019.40083 dblp:conf/icdar/CoquenetSCP19 fatcat:htj24bw6prbxbhhkb352ad3gbm

Cross-Lingual Pronoun Prediction with Deep Recurrent Neural Networks v2.0

Juhani Luotolahti, Jenna Kanerva, Filip Ginter
2017 Proceedings of the Third Workshop on Discourse in Machine Translation  
Our entry builds on our last year's success, our system based on deep recurrent neural networks outperformed all the other systems with a clear margin.  ...  This year we investigate whether different pre-trained word embeddings can be used to improve the neural systems, and whether the recently published Gated Convolutions outperform the Gated Recurrent Units  ...  We also experiment with changing the basic network units from Gated Recurrent Units to Gated Convolutions.  ... 
doi:10.18653/v1/w17-4808 dblp:conf/discomt/LuotolahtiKG17 fatcat:bsxtg2g3gjhqdkvtrwwn4jgbqa

Quasi-Recurrent Neural Networks [article]

James Bradbury, Stephen Merity, Caiming Xiong, Richard Socher
2016 arXiv   pre-print
We introduce quasi-recurrent neural networks (QRNNs), an approach to neural sequence modeling that alternates convolutional layers, which apply in parallel across timesteps, and a minimalist recurrent  ...  Experiments on language modeling, sentiment classification, and character-level neural machine translation demonstrate these advantages and underline the viability of QRNNs as a basic building block for  ...  As with convolutional neural networks, two or more QRNN layers should be stacked to create a model with the capacity to approximate more complex functions.  ... 
arXiv:1611.01576v2 fatcat:26zrh2glqfgqpol7qhd5d4ptja

An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling [article]

Shaojie Bai, J. Zico Kolter, Vladlen Koltun
2018 arXiv   pre-print
For most deep learning practitioners, sequence modeling is synonymous with recurrent networks.  ...  We conclude that the common association between sequence modeling and recurrent networks should be reconsidered, and convolutional networks should be regarded as a natural starting point for sequence modeling  ...  Gating Mechanisms One component that had been used in prior work on convolutional architectures for language modeling is the gated activation (van den Oord et al., 2016; Dauphin et al., 2017) .  ... 
arXiv:1803.01271v2 fatcat:ew45laqfazgstel5ls36s73wam

Layerwise Interweaving Convolutional LSTM [chapter]

Tiehang Duan, Sargur N. Srihari
2017 Lecture Notes in Computer Science  
A deep network structure is formed with LSTM layer and convolutional layer interweaves with each other.  ...  Its unique network structure allows it to extract higher level features with sequential information involved.  ...  The LIC-LSTM can serve as encoders in encoder-decoder model with natural language processing and neural machine translation tasks.  ... 
doi:10.1007/978-3-319-57351-9_31 fatcat:j7xcgjgbtnfr5f47gu56dwh5g4

Convolutional Gated MLP: Combining Convolutions gMLP [article]

A.Rajagopal, V. Nirmala
2021 arXiv   pre-print
Our implementation combines the best of Convolutional learning along with spatial gated MLP. Further, the paper visualizes how CgMLP learns.  ...  To the best of our knowledge, this is the first paper to introduce Convolutions to Gated MultiLayer Perceptron and contributes an implementation of this novel Deep Learning architecture.  ...  This allows for right sizing the neural network to open up the opportunity for tuning the network configuration for optimizing the generalization power of the model.  ... 
arXiv:2111.03940v1 fatcat:o7hmppmjzvbddmdveziexnheh4
« Previous Showing results 1 — 15 out of 20,438 results