Filters








22,575 Hits in 3.3 sec

Improved Attention Models for Memory Augmented Neural Network Adaptive Controllers [article]

Deepan Muthirayan, Scott Nivison, Pramod P. Khargonekar
2020 arXiv   pre-print
We introduced a working memory augmented adaptive controller in our recent work. The controller uses attention to read from and write to the working memory.  ...  In the above work, we used a soft-attention mechanism for the adaptive controller.  ...  In this work, we proposed a much improved attention mechanism for working memory augmented neural network adaptive controllers.  ... 
arXiv:1910.01189v7 fatcat:6xjsmsxbbjfgbmygkebccozu6y

Improving Neural Language Models with a Continuous Cache [article]

Edouard Grave, Armand Joulin, Nicolas Usunier
2016 arXiv   pre-print
We propose an extension to neural network language models to adapt their prediction to the recent history.  ...  We also draw a link between the use of external memory in neural network and cache models used with count based language models.  ...  Memory augmented neural networks.  ... 
arXiv:1612.04426v1 fatcat:op3qjnncvjethlvwvotvk75gmu

Memory-based Parameter Adaptation [article]

Pablo Sprechmann, Siddhant M. Jayakumar, Jack W. Rae, Alexander Pritzel, Adrià Puigdomènech Badia, Benigno Uria, Oriol Vinyals, Demis Hassabis, Razvan Pascanu, Charles Blundell
2018 arXiv   pre-print
Our method, Memory-based Parameter Adaptation, stores examples in memory and then uses a context-based lookup to directly modify the weights of a neural network.  ...  Much higher learning rates can be used for this local adaptation, reneging the need for many iterations over similar data before good predictions can be made.  ...  ACKNOWLEDGMENTS We would like to thank Gabor Melis for providing the LSTM baselines on the language tasks.  ... 
arXiv:1802.10542v1 fatcat:56h6yirgufabvixwp7cw5apgem

Working Memory Networks: Augmenting Memory Networks with a Relational Reasoning Module

Juan Pavez, Héctor Allende, Héctor Allende-Cid
2018 Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)  
To do that, models like Memory Networks (MemNNs) have combined external memory storages and attention mechanisms.  ...  During the last years, there has been a lot of interest in achieving some kind of complex reasoning using deep neural networks.  ...  Connections of the working memory model to memory augmented neural networks have been already studied in Graves et al. (2014) .  ... 
doi:10.18653/v1/p18-1092 dblp:conf/acl/PavezAA18 fatcat:ecc2tayjljgcrhygazxajpu244

Working Memory Networks: Augmenting Memory Networks with a Relational Reasoning Module [article]

Juan Pavez, Héctor Allende, Héctor Allende-Cid
2018 arXiv   pre-print
To do that, models like Memory Networks (MemNNs) have combined external memory storages and attention mechanisms.  ...  During the last years, there has been a lot of interest in achieving some kind of complex reasoning using deep neural networks.  ...  Connections of the working memory model to memory augmented neural networks have been already studied in Graves et al. (2014) .  ... 
arXiv:1805.09354v1 fatcat:qzqlac5tzfbsnbj6o5vq6mtqey

Memory Augmented Matching Networks for Few-Shot Learnings

Kien Tran, The authors are with the Department of Computer Science at National Defense Academy of Japan, Hiroshi Sato, Masao Kubo
2019 International Journal of Machine Learning and Computing  
Index Terms-Few shot learning, matching network, memory augmented neural network, prototypical network.  ...  With the advances in Deep learning field, meta-learning methods with deep networks could be better such as recurrent networks with enhanced external memory (Neural Turing Machines -NTM), metric learning  ...  Meta Learning with Memory Augmented Neural Network for One-Shot Learning Task Recent works have suggested the Memory Augmented Neural Network (MANN) for one-shot learning tasks via meta-learning approach  ... 
doi:10.18178/ijmlc.2019.9.6.867 fatcat:27wrqnpmorg6pmexvmmtoknllu

A short note on the decision tree based neural turing machine [article]

Yingshi Chen
2020 arXiv   pre-print
Neural turing machine(NTM) opens door for the memory network. It use differentiable attention mechanism to read/write external memory bank.  ...  The controller of RaDF is differentiable forest, the external memory of RaDF are response vectors which would be read/write by leaf nodes.  ...  NTM has different names in different papers, for example, memory augmented neural networks(MANN), reservoir memory machines, differentiable neural computer...  ... 
arXiv:2010.14753v1 fatcat:sf7flkl6z5gtjoo5xnhpzekzse

Neurocoder: Learning General-Purpose Computation Using Stored Neural Programs [article]

Hung Le, Svetha Venkatesh
2020 arXiv   pre-print
For the first time a Neural Program is treated as a datum in memory, paving the ways for modular, recursive and procedural neural programming.  ...  Artificial Neural Networks are uniquely adroit at machine learning by processing data through a network of artificial neurons.  ...  Memory Augmented Neural Networks (MANN) are an innovative solution allowing networks to access external memory for manipulating data [10, 11] .  ... 
arXiv:2009.11443v1 fatcat:55fx4dhcenfbjpmvyoepbgw4q4

Progressive Memory Banks for Incremental Domain Adaptation [article]

Nabiha Asghar, Lili Mou, Kira A. Selby, Kevin D. Pantasdo, Pascal Poupart, Xin Jiang
2020 arXiv   pre-print
We adopt the recurrent neural network (RNN) widely used in NLP, but augment it with a directly parameterized memory bank, which is retrieved by an attention mechanism at each step of RNN transition.  ...  Our model also outperforms previous work of IDA including elastic weight consolidation and progressive neural networks in the experiments.  ...  CONCLUSION In this paper, we propose a progressive memory network for incremental domain adaptation (IDA). We augment an RNN with an attention-based memory bank.  ... 
arXiv:1811.00239v2 fatcat:g2kg2klxf5eirn2p5d6wrqmjpy

Reasoning Over History: Context Aware Visual Dialog [article]

Muhammad A. Shah, Shikib Mehri, Tejas Srinivasan
2020 arXiv   pre-print
We extend the MAC network architecture with Context-aware Attention and Memory (CAM), which attends over control states in past dialog turns to determine the necessary reasoning operations for the current  ...  One strong VQA model is the MAC network, which decomposes a task into a series of attention-based reasoning steps.  ...  Conclusion We present Context-aware Attention and Memory (CAM), a set of dialog-specific augmentations to MAC networks (Hudson and Manning, 2018) .  ... 
arXiv:2011.00669v1 fatcat:ixchak7hafbofi4xvdo3g672pm

Automatic speech recognition for launch control center communication using recurrent neural networks with data augmentation and custom language model [article]

Kyongsik Yun, Joseph Osborne, Madison Lee, Thomas Lu, Edward Chow
2018 arXiv   pre-print
We showed that data augmentation and custom language models can improve speech recognition accuracy.  ...  We used bidirectional deep recurrent neural networks to train and test speech recognition performance.  ...  By using attention models in the future, we can bias the state of recurrent neural networks to improve contextbased speech recognition by paying more attention to specific words and phrases 26 .  ... 
arXiv:1804.09552v1 fatcat:li4nsatnk5etrbmbdqg4tvpxau

Dual Control Memory Augmented Neural Networks for Treatment Recommendations [article]

Hung Le, Truyen Tran, Svetha Venkatesh
2018 arXiv   pre-print
We approach the problem by using a memory-augmented neural network, in particular, by leveraging the recent differentiable neural computer that consists of a neural controller and an external memory module  ...  The resulting dual controller write-protected memory-augmented neural network is demonstrated on the MIMIC-III dataset on two tasks: procedure prediction and medication prescription.  ...  A DNC is an expressive recurrent neural network consisting of a controller augmented with a memory module.  ... 
arXiv:1802.03689v1 fatcat:65rduimdzbg4tc57bxhvpndjlq

Neural Stored-program Memory [article]

Hung Le, Truyen Tran, Svetha Venkatesh
2019 arXiv   pre-print
The proposed model, dubbed Neural Stored-program Memory, augments current memory-augmented neural networks, creating differentiable machines that can switch programs through time, adapt to variable contexts  ...  Neural networks powered with external memory simulate computer behaviors. These models, which use the memory to store data for a neural controller, can learn algorithms and other complex tasks.  ...  These findings have sparked a new research direction called Memory Augmented Neural Networks (MANNs) that emulate modern computer behavior by detaching memorization from computation via memory and controller  ... 
arXiv:1906.08862v2 fatcat:yaf4iykmirar3bx6cuxuvf2ilu

Improving Long Handwritten Text Line Recognition with Convolutional Multi-way Associative Memory [article]

Duc Nguyen, Nhan Tran, Hung Le
2020 arXiv   pre-print
Inspired by recently proposed memory-augmented neural networks (MANNs) for long-term sequential modeling, we present a new architecture dubbed Convolutional Multi-way Associative Memory (CMAM) to tackle  ...  Convolutional Recurrent Neural Networks (CRNNs) excel at scene text recognition.  ...  In this work, we adapt recent memory-augmented neural networks by integrating an external memory module into a convolutional neural network.  ... 
arXiv:1911.01577v2 fatcat:i6xuxvxoand53gt55uckaxhiv4

Text Normalization using Memory Augmented Neural Networks

Subhojeet Pramanik, Aman Hussain
2019 Speech Communication  
We perform text normalization, i.e. the transformation of words from the written to the spoken form, using a memory augmented neural network.  ...  errors made by the LSTM-based recurrent neural networks.  ...  Acknowledgements We would like to show our gratitude to Richard Sproat, Research Scientist at Google Brain, for his insights and comments that greatly improved the manuscript.  ... 
doi:10.1016/j.specom.2019.02.003 fatcat:hl2xnfoncbbfte5fu32vz4syie
« Previous Showing results 1 — 15 out of 22,575 results