Filters








11,137 Hits in 1.7 sec

Accelerating recurrent neural network training using sequence bucketing and multi-GPU data parallelization [article]

Viacheslav Khomenko , Kostiantyn Bokhan
2017 arXiv   pre-print
An example is given for the online handwriting recognition task using an LSTM recurrent neural network.  ...  An efficient algorithm for recurrent neural network training is presented. The approach increases the training speed for tasks where a length of the input sequence may vary significantly.  ...  Bokhan, "Accelerating recurrent neural network training using sequence bucketing and multi-GPU data parallelization," 2016 IEEE First International Conference on Data Stream Mining & Processing (DSMP),  ... 
arXiv:1708.05604v1 fatcat:rzjhroipufeo3bhqrvoz7co424

Sliding Window and Parallel LSTM with Attention and CNN for Sentence Alignment on Low-Resource Languages

Tien-Ping Tan, Chai Kim Lim, Wan Rose Eliza Abdul Rahman
2021 Pertanika journal of science & technology  
The classification accuracy of these models was evaluated using Malay-English parallel text corpus and UN French-English parallel text corpus.  ...  This paper proposes a parallel long-short-term memory with attention and convolutional neural network (parallel LSTM+Attention+CNN) for classifying two sentences as parallel or non-parallel sentences.  ...  Grégoire and Langlais (2017) proposed to use bidirectional recurrent neural networks to extract parallel sentences from Wikipedia.  ... 
doi:10.47836/pjst.30.1.06 fatcat:f6kyvguvsvba3n57hul6aoj7li

Extracting Parallel Sentences from Nonparallel Corpora Using Parallel Hierarchical Attention Network

Shaolin Zhu, Yong Yang, Chun Xu
2020 Computational Intelligence and Neuroscience  
To address this issue, we propose a novel parallel hierarchical attention neural network which encodes monolingual sentences versus bilingual sentences and construct a classifier to extract parallel sentences  ...  In particular, our attention mechanism structure can learn different alignment weights of words in parallel sentences.  ...  More recent approaches used deep learning, such as convolutional neural networks [13] and recurrent neural networks based on long short-term memory (LSTM) [1, 14, 15] to learn an end-to-end network  ... 
doi:10.1155/2020/8823906 pmid:32952544 pmcid:PMC7482026 fatcat:lv2fskl5lrernbcmaisa4g3vqy

End-to-End Sequence Labeling via Convolutional Recurrent Neural Network with a Connectionist Temporal Classification Layer

Xiaohui Huang, Lisheng Qiao, Wentao Yu, Jing Li, Yanzhou Ma
2020 International Journal of Computational Intelligence Systems  
In this paper, we propose a kind of novel deep neural network architecture which combines convolution, pooling and recurrent in a unified framework to construct the convolutional recurrent neural network  ...  of long input to short output which will also reduce he model's complexity, and adopt Connectionist Temporal Classification (CTC) layer to achieve an end-to-end pattern for sequence labeling.  ...  ACKNOWLEDGMENTS The authors would like to thank the anonymous reviewers for the constructive comments, and this work is sponsored by the National key research and development program of China (No. 2016YFB0201402  ... 
doi:10.2991/ijcis.d.200316.001 fatcat:pdsqjcacj5du7p4l4tzdlfdgsq

DNA Sequences Classification with Deep Learning: A Survey

Samia M. Abd –Alhalem, El-Sayed M. El-Rabaie, Naglaa. F. Soliman, Salah Eldin S. E. Abdulrahman, Nabil A. Ismail, Fathi E. Abd El-samie
2021 Menoufia Journal of Electronic Engineering Research  
neural networks to hyper parameter tuning, and the most recent state-of-the-art DL architectures used in DNA classification.  ...  In This work, we start from the previous classification methods such as alignment methods pointing out the problems, which are face to use these methods.After that, we demonstrate deep learning, from artificial  ...  Then introduce the components of the convolutional neural networks (CNNs) and recurrent neural networks (RNNs) that mainly used for DNA sequences classification.  ... 
doi:10.21608/mjeer.2021.146090 fatcat:vkfpn7wb3bfqtdmxer2cli3tqu

Inducing Multilingual Text Analysis Tools Using Bidirectional Recurrent Neural Networks [article]

Othman Zennaki and Nasredine Semmar and Laurent Besacier
2016 arXiv   pre-print
We experiment several cross-lingual annotation projection methods using Recurrent Neural Networks (RNN) models.  ...  We demonstrate the validity and genericity of our model by using parallel corpora (obtained by manual or automatic translation).  ...  Recurrent Neural Networks There are two major architectures of neural networks: Feedforward (Bengio et al., 2003) and Recurrent Neural Networks (RNN) (Schmidhuber, 1992; Mikolov et al., 2010) .  ... 
arXiv:1609.09382v1 fatcat:fx7mxgjrzndatkupa5noxvgcue

Recurrent Neural Network Method in Arabic Words Recognition System [article]

Yusuf Perwej
2013 arXiv   pre-print
The key innovation is a recently produce recurrent neural networks objective function known as connectionist temporal classification.  ...  The system consists of an advanced recurrent neural network with an output layer designed for sequence labeling, partially combined with a probabilistic language model.  ...  Connectionist temporal classification (CTC) is an objective function designed for sequence labeling with recurrent neural network.  ... 
arXiv:1301.4662v1 fatcat:bkovhn6bpvandkpl6qnyo7eob4

A Character-Level Decoder without Explicit Segmentation for Neural Machine Translation [article]

Junyoung Chung, Kyunghyun Cho, Yoshua Bengio
2016 arXiv   pre-print
To answer this question, we evaluate an attention-based encoder-decoder with a subword-level encoder and a character-level decoder on four language pairs--En-Cs, En-De, En-Ru and En-Fi-- using the parallel  ...  In this paper, we ask a fundamental question: can neural machine translation generate a character sequence without any explicit segmentation?  ...  We acknowledge the support of the following agencies for research funding and computing support: NSERC, Calcul Québec, Compute Canada, the Canada Research Chairs, CIFAR and Samsung.  ... 
arXiv:1603.06147v4 fatcat:3te3arnafved5n5kx3jq342spu

Simple Recurrent Units for Highly Parallelizable Recurrence

Tao Lei, Yu Zhang, Sida I. Wang, Hui Dai, Yoav Artzi
2018 Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing  
Common recurrent neural architectures scale poorly due to the intrinsic difficulty in parallelizing their state computations.  ...  SRU achieves 5-9x speed-up over cuDNN-optimized LSTM on classification and question answering datasets, and delivers stronger results than LSTM and convolutional models.  ...  We thank Adam Yala, Howard Chen, Jeremy Wohlwend, Lili Yu, Kyle Swanson and Kevin Yang for providing useful feedback on the paper and the SRU implementation.  ... 
doi:10.18653/v1/d18-1477 dblp:conf/emnlp/LeiZWDA18 fatcat:55hgrm6vjjbejanbzk5o435bke

A Character-level Decoder without Explicit Segmentation for Neural Machine Translation

Junyoung Chung, Kyunghyun Cho, Yoshua Bengio
2016 Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)  
In this paper, we ask a fundamental question: can neural machine translation generate a character sequence without any explicit segmentation?  ...  To answer this question, we evaluate an attention-based encoderdecoder with a subword-level encoder and a character-level decoder on four language pairs-En-Cs, En-De, En-Ru and En-Fiusing the parallel  ...  We acknowledge the support of the following agencies for research funding and computing support: NSERC, Calcul Québec, Compute Canada, the Canada Research Chairs, CIFAR and Samsung.  ... 
doi:10.18653/v1/p16-1160 dblp:conf/acl/ChungCB16 fatcat:ub5hlcjs5zbcbbyxytes3aav4y

Symbol Grounding in Multimodal Sequences using Recurrent Neural Networks

Federico Raue, Wonmin Byeon, Thomas M. Breuel, Marcus Liwicki
2015 Neural Information Processing Systems  
Our approach uses two Long Short-Term Memory (LSTM) networks for multimodal sequence learning and recovers the internal symbolic space using an EM-style algorithm.  ...  A priori, both visual inputs and auditory inputs are complex analog signals with a large amount of noise and context, and lacking of any segmentation information.  ...  gradient in recurrent neural networks [10, 11] .  ... 
dblp:conf/nips/RaueBBL15 fatcat:drx7z7oatjailp4rlbgs2ya2hy

Multilingual Hierarchical Attention Networks for Document Classification

Nikolaos Pappas, Andrei Popescu-Belis
2017 Zenodo  
To this end, we propose multilingual hierarchical attention networks for learning document structures, with shared encoders and/or attention mechanisms across languages, using multi-task learning and an  ...  Hierarchical attention networks have recently achieved remarkable performance for document classification in a given language.  ...  Introduction Learning word sequence representations has become increasingly useful for a variety of NLP tasks such as document classification (Tang et al., 2015; Yang et al., 2016) , neural machine translation  ... 
doi:10.5281/zenodo.834306 fatcat:42qjk2f5jnaexcsij7hleyh4wq

Grapheme-to-phoneme conversion using Long Short-Term Memory recurrent neural networks

Kanishka Rao, Fuchun Peng, Hasim Sak, Francoise Beaufays
2015 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)  
We propose a G2P model based on a Long Short-Term Memory (LSTM) recurrent neural network (RNN).  ...  Training joint-sequence based G2P require explicit graphemeto-phoneme alignments which are not straightforward since graphemes and phonemes don't correspond one-to-one.  ...  In this paper we present a novel approach to the problem using Long Short-Term Memory (LSTM) neural networks [3] which are a class of recurrent neural network especially suited for sequence modeling.  ... 
doi:10.1109/icassp.2015.7178767 dblp:conf/icassp/RaoPSB15 fatcat:rzn5y3jd3jbghknshsktk24wee

Online Sequence Training of Recurrent Neural Networks with Connectionist Temporal Classification [article]

Kyuyeon Hwang, Wonyong Sung
2017 arXiv   pre-print
Connectionist temporal classification (CTC) based supervised sequence training of recurrent neural networks (RNNs) has shown great success in many machine learning areas including end-to-end speech and  ...  Furthermore, the length of training sequences is usually not uniform, which makes parallel training with multiple sequences inefficient on shared memory models such as graphics processing units (GPUs).  ...  When combined with recurrent neural networks (RNNs), supervised sequence learning has shown great success in many applications including machine translation Sutskever et al., 2014; Cho et al., 2014) ,  ... 
arXiv:1511.06841v5 fatcat:rmznkad35bctbnjfb2pdjefelu

Hybrid Machine Translation with Multi-Source Encoder-Decoder Long Short-Term Memory in English-Malay Translation

Yin-Lai Yeong, Tien-Ping Tan, Keng Hoon Gan, Siti Khaotijah Mohammad
2018 International Journal on Advanced Science, Engineering and Information Technology  
The translation produced by an SMT is based on the statistical analysis of text corpora, while NMT uses the deep neural network to model and to generate a translation.  ...  SMT and NMT have their strength and weaknesses. SMT may produce a better translation with a small parallel text corpus compared to NMT.  ...  The success is due to the introduction and advancement of approaches in convolutional neural networks (CNN) and recurrent neural networks (RNN) that models very complex patterns using deep layers of neural  ... 
doi:10.18517/ijaseit.8.4-2.6816 fatcat:uh4nvy3tkfailgxxt7ux4j2yp4
« Previous Showing results 1 — 15 out of 11,137 results