44,515 Hits in 4.6 sec

Learning Stochastic Recurrent Networks [article]

Justin Bayer, Christian Osendorfer
2015 arXiv   pre-print
Leveraging advances in variational inference, we propose to enhance recurrent neural networks with latent variables, resulting in Stochastic Recurrent Networks (STORNs).  ...  is a generalisation of deterministic recurrent neural networks.  ...  We will first describe the used model family, that of recurrent neural networks and then the estimator, stochastic gradient variational Bayes (SGVB).  ... 
arXiv:1411.7610v3 fatcat:vdft7yrvznhvbcb3w4iggbbkay

Stochastic variational learning in recurrent spiking networks

Danilo Jimenez Rezende, Wulfram Gerstner
2014 Frontiers in Computational Neuroscience  
Learning in Recurrent spiking networks Frontiers in Computational Neuroscience  ...  Here we derive and investigate a new learning rule for recurrent spiking networks with hidden neurons, combining principles from variational learning and reinforcement learning.  ...  by recurrent spiking networks.  ... 
doi:10.3389/fncom.2014.00038 pmid:24772078 pmcid:PMC3983494 fatcat:lj2xei7sjneyzmrn7ix6fczgka

Slow stochastic Hebbian learning of classes of stimuli in a recurrent neural network

Nicolas Brunel, Francesco Carusi, Stefano Fusi
1998 Network  
We study unsupervised Hebbian learning in a recurrent network in which synapses have a finite number of stable states.  ...  of activity correlated with the stimulus to become an attractor of the recurrent network.  ...  In such models, the presentation of a stimulus to a recurrent network provokes modifications in the efficacy of recurrent collaterals.  ... 
doi:10.1088/0954-898x/9/1/007 pmid:9861982 fatcat:jniqvmnganhf7aa2rqkk4lhtey

Slow stochastic Hebbian learning of classes of stimuli in a recurrent neural network

Nicolas Brunel, Francesco Carusi, Stefano Fusi
1998 Network  
We study unsupervised Hebbian learning in a recurrent network in which synapses have a finite number of stable states.  ...  of activity correlated with the stimulus to become an attractor of the recurrent network.  ...  In such models, the presentation of a stimulus to a recurrent network provokes modifications in the efficacy of recurrent collaterals.  ... 
doi:10.1088/0954-898x_9_1_007 pmid:9861982 fatcat:7tsgzhjvf5hmzni5m5kbi2jz7e

Dynamic Scheduling for Stochastic Edge-Cloud Computing Environments using A3C learning and Residual Recurrent Neural Networks

Shreshth Tuli, Shashikant Ilager, Kotagiri Ramamohanarao, Rajkumar Buyya
2020 IEEE Transactions on Mobile Computing  
However, Asynchronous-Advantage-Actor-Critic (A3C) learning is known to quickly adapt to dynamic scenarios with less data and Residual Recurrent Neural Network (R2N2) to quickly update model parameters  ...  Thus, we propose an A3C based real-time scheduler for stochastic Edge-Cloud environments allowing decentralized learning, concurrently across multiple agents.  ...  ACKNOWLEDGEMENTS This research work is supported by the Melbourne-Chindia Cloud Computing (MC3) Research Network and the Australian Research Council.  ... 
doi:10.1109/tmc.2020.3017079 fatcat:ubmsvqg5anfplaaiovazrijp4a

Evaluating Neuromodulator-controlled Stochastic Plasticity for Learning Recurrent Neural Control Networks

Christian W. Rempis, Hazem Toutounji, Frank Pasemann
2013 Proceedings of the 5th International Joint Conference on Computational Intelligence  
Learning recurrent neural networks as behavior controllers for robots requires measures to guide the learning towards a desired behavior.  ...  The performance depends strongly on the complexity of the task and less on the chosen network topology.  ...  In this domain, efficient learning methods are rare due to the difficulties inherent to learning in recurrent neural networks.  ... 
doi:10.5220/0004554504890496 dblp:conf/ijcci/RempisTP13 fatcat:theesgql2jcmvmtsvwhgqwl2qm

Stochastic Convolutional Recurrent Networks for Language Modeling

Jen-Tzung Chien, Yu-Min Huang
2020 Interspeech 2020  
Sequential learning using recurrent neural network (RNN) has been popularly developed for language modeling.  ...  Experiments on language modeling demonstrate the effectiveness of stochastic convolutional recurrent network relative to the other sequential machines in terms of perplexity and word error rate.  ...  Sequential Learning This paper constructs a new neural network structure which integrates convolutional neural network and recurrent neural network for sequential learning.  ... 
doi:10.21437/interspeech.2020-1493 dblp:conf/interspeech/ChienH20 fatcat:o6chawsxxbcwncx5t4i2gg4cbe

Recurrent Attentive Neural Process for Sequential Data [article]

Shenghao Qin, Jiacheng Zhu, Jimmy Qin, Wenshuo Wang, Ding Zhao
2019 arXiv   pre-print
Neural processes (NPs) learn stochastic processes and predict the distribution of target output adaptively conditioned on a context set of observed input-output pairs.  ...  In this paper, we proposed the Recurrent Attentive Neural Process (RANP), or alternatively, Attentive Neural Process-RecurrentNeural Network(ANP-RNN), in which the ANP is incorporated into a recurrent  ...  Related Works Recently, there has been a increasing interest in learning and inferring stochastic processes with neural networks.  ... 
arXiv:1910.09323v1 fatcat:kojeha3ih5hvhp3crwqx6avsn4

An Emergent Learning Method for Recurrent Neural Networks

Ken KITAGAWA, Noriyasu HONMA, Kenichi ABE
1997 Transactions of the Society of Instrument and Control Engineers  
We propose a learning method for recurrent neural networks with dynamics. The core of this method is to keep a complexity of the network dynamics in the vicinity of the edge of chaos.  ...  Key Words: recurrent neural networks, chaos, complex system, learning xi(t+1)=1,(1)1+ exp(-aisi(t)) Ns(t)=Lwijxj(t)+wiuui(t)+ei,(2) y(t)=(t=1,2,(3)  ... 
doi:10.9746/sicetr1965.33.1093 fatcat:g7wxrqg3zbh5le4pqrgplnaymy

Deep Bayesian Natural Language Processing

Jen-Tzung Chien
2019 Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: Tutorial Abstracts  
stochastic temporal convolutional net stochastic recurrent neural network regularized recurrent neural network stochastic learning & normalizing flows -VAE with VampPrior skip recurrent neural network  ...  Z-forcing: Training stochastic recurrent networks. In Advances in Neural Information Processing Systems 30, pages 6713-6723.  ... 
doi:10.18653/v1/p19-4006 dblp:conf/acl/Chien19 fatcat:bj6qf6cpkffz3oxinswh5fy4ry

Dual stochasticity of neurons and synapses provides a biologically plausible learning [article]

Jun-nosuke Teramae
2019 bioRxiv   pre-print
Neurons and synapses in the cerebral cortex behave stochastically even during precise perception and reliable learning of animals.  ...  The algorithm can be regarded as an integration of major branches of existing algorithms of neural networks, back propagation learning and Boltzmann machine, while it overcomes known limitations of them  ...  Figure 3 . 3 Supervised learning of a recurrent network on the MNIST dataset.  ... 
doi:10.1101/811646 fatcat:n2hhlu6xpbhl7enumprmkuky6u

Memory-based control with recurrent neural networks [article]

Nicolas Heess, Jonathan J Hunt, Timothy P Lillicrap, David Silver
2015 arXiv   pre-print
We extend two related, model-free algorithms for continuous control -- deterministic policy gradient and stochastic value gradient -- to solve partially observed domains using recurrent neural networks  ...  We find that recurrent deterministic and stochastic policies are able to learn similarly good solutions to these tasks, including the water maze where the agent must learn effective search strategies.  ...  critic but learns a stochastic policy.  ... 
arXiv:1512.04455v1 fatcat:slnul32zezfsvlkl73iwecjjwi

Benchmarking Deep Sequential Models on Volatility Predictions for Financial Time Series [article]

Qiang Zhang, Rui Luo, Yaodong Yang, Yuanyuan Liu
2018 arXiv   pre-print
We examine both the traditional approaches and the deep sequential models on the task of volatility prediction, including the most recent variants of convolutional and recurrent networks, such as the dilated  ...  models, including dilated CNN and Dilated RNN, produce most accurate estimation and prediction, outperforming various widely-used deterministic models in the GARCH family and several recently proposed stochastic  ...  the stochastic volatility model with stochastic recurrent neural networks and variational inference by Luo et al.  ... 
arXiv:1811.03711v1 fatcat:3nv7kfctm5fbhljeoptwrbe74q

Introduction to the theory of neural computation

1994 Neural Networks  
6.5 A Theoretical Framework for Generalization 147 6.6 Optimal Network Architectures 156 SEVEN Recurrent Networks 163 7.1 Boltzmann Machines 163 7.2 Recurrent Back-Propagation 172 7.3 Learning  ...  Networks 32 2.5 Capacity of the Stochastic Network 35 THREE Extensions of the Hopfield Model 43 3.1 Variations an the Hopfield Model 43 3.2 Correlated Patterns 49 3.3 Continuous-Valued  ... 
doi:10.1016/0893-6080(94)90014-0 fatcat:yniof2br4nfydkj2wkht6z356e


Alvin Rindra Fazrie
Echo State Networks adalah salah satu arsitektur dari Jaringan Syaraf Tiruan yang berdasarkan prinsip Supervised Learning untuk Recurrent Neural Network, dieksplorasi bersama Stochastic-learning Grammar  ...  In this paper, Echo State Networks and Stochastic-learning grammar are explored in order to get an idea about generating human's natural language and the possibilities of integrating these methods to make  ...  Echo state network is an architecture to supervise the learning principle to generate randomly recurrent neural networks [3] .  ... 
doi:10.15408/jti.v11i1.6093 fatcat:3udwq22xjbe65gtmepjr3qhxoy
« Previous Showing results 1 — 15 out of 44,515 results