Filters








830 Hits in 10.2 sec

Low-pass Recurrent Neural Networks - A memory architecture for longer-term correlation discovery [article]

Thomas Stepleton, Razvan Pascanu, Will Dabney, Siddhant M. Jayakumar, Hubert Soyer, Remi Munos
2018 arXiv   pre-print
Besides memory demands, learning dynamics like vanishing gradients and slow convergence due to infrequent weight updates can reduce BPTT's practicality; meanwhile, although online recurrent network learning  ...  This is especially true during the initial learning stages, when exploratory behaviour can increase the delay between specific actions and their effects.  ...  LSTM and GRU architectures introduce gating mechanisms to mitigate vanishing gradients, and constraining recurrent network weights can also help.  ... 
arXiv:1805.04955v1 fatcat:wd2gnhpuxjckvexvwl3yzz3pnm

Application of deep learning algorithms and architectures in the new generation of mobile networks

Dejan Dasic, Miljan Vucetic, Nemanja Ilic, Milos Stankovic, Marko Beko
2021 Serbian Journal of Electrical Engineering  
Finally, the paper presents practical use case of modulation classification as implementation of deep learning in an application essential for modern spectrum management.  ...  Having firstly presented the background of deep learning and related technologies, the paper goes on to present the architectures used for deployment of deep learning in mobile networks.  ...  In [18] the authors proposed networks with Gated Recurrent Units (GRUs), representing, roughly speaking, light and trimmed LSTM alternative, showing that complex architecture of LSTMs may not be necessary  ... 
doi:10.2298/sjee2103397d fatcat:n3hduljspfbt3mkq2zdzbae72u

RMDL

Kamran Kowsari, Mojtaba Heidarysafa, Donald E. Brown, Kiana Jafari Meimandi, Laura E. Barnes
2018 Proceedings of the 2nd International Conference on Information System and Data Mining - ICISDM '18  
This paper introduces Random Multimodel Deep Learning (RMDL): a new ensemble, deep learning approach for classification.  ...  This paper describes RMDL and shows test results for image and text data including MNIST, CIFAR-10, WOS, Reuters, IMDB, and 20newsgroup.  ...  RMDL uses two specific RNN structures: Gated Recurrent Units (GRUs) and Long Short-Term Memory (LSTM).  ... 
doi:10.1145/3206098.3206111 dblp:conf/icisdm/KowsariHBMB18 fatcat:lzyx7ze67vgu5lv2jbguion544

Recent Advances in Recurrent Neural Networks [article]

Hojjat Salehinejad, Sharan Sankar, Joseph Barfett, Errol Colak, Shahrokh Valaee
2018 arXiv   pre-print
Recurrent neural networks (RNNs) are capable of learning features and long term dependencies from sequential and time-series data.  ...  A well-trained RNN can model any dynamical system; however, training RNNs is mostly plagued by issues in learning long-term dependencies.  ...  In [94] , RCNNs are used for text classification on several datasets.  ... 
arXiv:1801.01078v3 fatcat:ioxziqbkmzdrfoh2kukul6xlku

Long-Term Recurrent Convolutional Networks for Visual Recognition and Description

Jeff Donahue, Lisa Anne Hendricks, Marcus Rohrbach, Subhashini Venugopalan, Sergio Guadarrama, Kate Saenko, Trevor Darrell
2017 IEEE Transactions on Pattern Analysis and Machine Intelligence  
We develop a novel recurrent convolutional architecture suitable for large-scale visual learning which is end-to-end trainable, and demonstrate the value of these models on benchmark video recognition  ...  Long-term RNN models are appealing in that they directly can map variable-length inputs (e.g., video frames) to variable length outputs (e.g., natural language text) and can model complex temporal dynamics  ...  This work was supported in part by DARPA's MSEE and SMISC programs, NSF awards IIS-1427425, and IIS-1212798, IIS-1116411, Toyota, and the Berkeley Vision and Learning Center.  ... 
doi:10.1109/tpami.2016.2599174 pmid:27608449 fatcat:xoluu2wo6jfbxddalikzfbrgk4

Long-term recurrent convolutional networks for visual recognition and description

Jeff Donahue, Lisa Anne Hendricks, Sergio Guadarrama, Marcus Rohrbach, Subhashini Venugopalan, Trevor Darrell, Kate Saenko
2015 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)  
We develop a novel recurrent convolutional architecture suitable for large-scale visual learning which is end-to-end trainable, and demonstrate the value of these models on benchmark video recognition  ...  Long-term RNN models are appealing in that they directly can map variable-length inputs (e.g., video frames) to variable length outputs (e.g., natural language text) and can model complex temporal dynamics  ...  This work was supported in part by DARPA's MSEE and SMISC programs, NSF awards IIS-1427425, and IIS-1212798, IIS-1116411, Toyota, and the Berkeley Vision and Learning Center.  ... 
doi:10.1109/cvpr.2015.7298878 dblp:conf/cvpr/DonahueHGRVDS15 fatcat:5w4eeyesm5hipiieav2nzc4et4

DeepFuseOSV: online signature verification using hybrid feature fusion and depthwise separable convolution neural network architecture

Chandra Sekhar Vorugunti, Viswanath Pulabaigari, Prerana Mukherjee, Abhishek Sharma
2020 IET Biometrics  
DWSCNN is utilised for extracting deep feature representations and LSTM is competent in learning long term dependencies of stroke points of a signature.  ...  (ii) a hybrid architecture combining depth-wise separable convolution neural network (DWSCNN) and long short term memory (LSTM) network delivering state-of-the-art performance for OSV is proposed.  ...  [21] structured a novel semisupervised system for assessing the signature complexity utilising recurrent neural networks (RNNs).  ... 
doi:10.1049/iet-bmt.2020.0032 fatcat:vhhcmpakz5c4di3o7tbkfcaqsa

On Architectures for Including Visual Information in Neural Language Models for Image Description [article]

Marc Tanti and Albert Gatt and Kenneth P. Camilleri
2019 arXiv   pre-print
We also observe that the merge architecture can have its recurrent neural network pre-trained in a text-only language model (transfer learning) rather than be initialised randomly as usual.  ...  Our work opens up new avenues of research in neural architectures, explainable AI, and transfer learning.  ...  compared architectures with simple RNNs and LSTMs.  ... 
arXiv:1911.03738v1 fatcat:ncbx3ee22nhmfjh4u6mierrrbu

Systematic reviews in sentiment analysis: a tertiary study

Alexander Ligthart, Cagatay Catal, Bedir Tekinerdogan
2021 Artificial Intelligence Review  
According to this analysis, LSTM and CNN algorithms are the most used deep learning algorithms for sentiment analysis.  ...  In addition to the tertiary study, we also identified recent 112 deep learning-based sentiment analysis papers and categorized them based on the applied deep learning algorithms.  ...  The advantages of Recurrent Neural Networks are their simple architectures and their ability to learn tree structures.  ... 
doi:10.1007/s10462-021-09973-3 fatcat:zo7igc4fnnh47kyafncbfmaf3u

RC-RNN: Reconfigurable Cache Architecture for Storage Systems Using Recurrent Neural Networks

Shahriar Ebrahimi, Reza Salkhordeh, Seyed Ali Osia, Ali Taheri, Hamid R. Rabiee, Hossein Asadi
2021 IEEE Transactions on Emerging Topics in Computing  
In this paper, we propose RC-RNN, the first reconfigurable SSD-based cache architecture for storage systems that utilizes machine learning to identify performance-critical data pages for I/O caching.  ...  The proposed architecture uses Recurrent Neural Networks (RNN) to characterize ongoing workloads and optimize itself towards higher cache performance while improving SSD lifetime.  ...  We utilize Recurrent Neural Network (RNN) in the proposed architecture as one of the most powerful machine learning methods, which is proven to be accurate in several application domains such as text analysis  ... 
doi:10.1109/tetc.2021.3102041 fatcat:gyqnb3526fbebohwr4qapeiche

Hybrid deep neural network for Bangla automated image descriptor

Md Asifuzzaman Jishan, Khan Raqib Mahmud, Abul Kalam Al Azad, Md Shahabub Alam, Anif Minhaz Khan
2020 IJAIN (International Journal of Advances in Intelligent Informatics)  
The model employs Convolutional Neural Network (CNN) to classify the whole dataset, while Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) capture the sequential semantic representation  ...  In this study, a novel dataset was constructed by generating Bangla textual descriptor from visual input, called Bangla Natural Language Image to Text (BNLIT), incorporating 100 classes with annotation  ...  Acknowledgment The authors are thankful to the Department of Computer Science and Engineering at the University of Liberal Arts Bangladesh for their research lab and other crucial technical support along  ... 
doi:10.26555/ijain.v6i2.499 fatcat:5wamb5fycjgtvai3dqhd33yq3a

Online handwritten Gurmukhi word recognition using fine-tuned Deep Convolutional Neural Network on offline features

Sukhdeep Singh, Anuj Sharma, Vinod Kumar Chauhan
2021 Machine Learning with Applications  
The present study provided benchmark results for online handwritten Gurmukhi word recognition using deep learning architecture convolutional neural network, and obtained above 97% recognition accuracy  ...  Experimental results demonstrated that the deep learning system achieved great results in Gurmukhi script and outperforms existing results in the literature.  ...  These controlled states are referred as gated memory/state, and these are part of LSTMs and gated recurrent units.  ... 
doi:10.1016/j.mlwa.2021.100037 fatcat:ohu2dktiifft3lyecf6g3yuauu

CNN Variants for Computer Vision: History, Architecture, Application, Challenges and Future Scope

Dulari Bhatt, Chirag Patel, Hardik Talsania, Jigar Patel, Rasmika Vaghela, Sharnil Pandya, Kirit Modi, Hemant Ghayvat
2021 Electronics  
Several inspirational concepts for the progress of CNN have been investigated, including alternative activation functions, regularization, parameter optimization, and architectural advances.  ...  Furthermore, the introduction of large amounts of data and readily available hardware has opened new avenues for CNN study.  ...  Acknowledgments: The authors would like to thank the reviewers for their valuable suggestions which helped in improving the quality of this paper.  ... 
doi:10.3390/electronics10202470 fatcat:aqhrysjtbjagzl6byalgy2du5a

Framework for Deep Learning-Based Language Models using Multi-task Learning in Natural Language Understanding: A Systematic Literature Review and Future Directions

Rahul Manohar Samant, Mrinal Bachute, Shilpa Gite, Ketan Kotecha
2022 IEEE Access  
NLU (Natural Language Understanding) is a subset of NLP including tasks, like machine translation, dialogue-based systems, natural language inference, text entailment, sentiment analysis, etc.  ...  INDEX TERMS Deep learning, Knowledge representation, Multi-task NLU, Unsupervised learning TABLE I APPLICATION DOMAINS FOR NLU Domain Applications Machine translation IBM Watson Task-based dialogue-based  ...  MT-LSTM is reported to beat a collection of baselines, including LSTM and RNN-based text categorization models.  ... 
doi:10.1109/access.2022.3149798 fatcat:k3kdt4eryzdfpk5k6w62jtlskm

A tutorial survey of architectures, algorithms, and applications for deep learning

Li Deng
2014 APSIPA Transactions on Signal and Information Processing  
classification and for feature learning.  ...  Three representative deep architectures -deep autoencoders, deep stacking networks with their generalization to the temporal domain (recurrent networks), and deep neural networks (pretrained with deep  ...  Although such more complex architectures have produced state of the a tutorial survey of architectures, algorithms, and applications for deep learning art results (e.g., [154] ), their complexity does  ... 
doi:10.1017/atsip.2013.9 fatcat:4l4uonhhcffkbfot2fztpfxo2e
« Previous Showing results 1 — 15 out of 830 results