20,654 Hits in 5.1 sec

Learning Intrinsic Sparse Structures within Long Short-Term Memory [article]

Wei Wen, Yuxiong He, Samyam Rajbhandari, Minjia Zhang, Wenhan Wang, Fang Liu, Bin Hu, Yiran Chen, Hai Li
2018 arXiv   pre-print
This work aims to learn structurally-sparse Long Short-Term Memory (LSTM) by reducing the sizes of basic structures within LSTM units, including input updates, gates, hidden states, cell states and outputs  ...  To overcome the problem, we propose Intrinsic Sparse Structures (ISS) in LSTMs.  ...  RELATED WORK LEARNING INTRINSIC SPARSE STRUCTURES INTRINSIC SPARSE STRUCTURES The computation within LSTMs is (Hochreiter & Schmidhuber (1997) ) i t = σ (x t · W xi + h t−1 · W hi + b i ) f t = σ  ... 
arXiv:1709.05027v7 fatcat:klmh7fcrvzhgfcfuv3jn2yszme

Central processing in the mushroom bodies

Mark Stopfer
2014 Current Opinion in Insect Science  
and memory.  ...  Here, focusing mainly on olfaction, we discuss functionally related roles the mushroom bodies play in signal gain control, response sparsening, the separation of similar signals (decorrelation), and learning  ...  Right: Short-term memory (stm), medium-term memory (mtm) and long-term memory (ltm) are processed in the γ, α'/β', and α/β lobes, respectively.  ... 
doi:10.1016/j.cois.2014.10.009 pmid:25621203 pmcid:PMC4303581 fatcat:433zmwy5szdfvfm2jciljpq5xu

Functional and comparative assessements of the octopus learning and memory system

Binyamin Hochner
2010 Frontiers in bioscience (Scholar edition)  
Long term potentiation (LTP) of the MSF-VL synaptic input 6. How the VL and its LTP are involved in behavioral learning and memory? 7. Summary 7.1.  ...  The learning and memory system of Octopus vulgaris 5.1. Morphological organization of the octopus VL-MSF syste 5.2. Electrophysiological characterization of the MSF and VL neurons 5.3.  ...  (39) showed that, in the octopus, short-term memory is stored outside the brain area that uses LTP to acquire long-term memory (the VL), since short-term learning during training for the avoidance task  ... 
doi:10.2741/s99 pmid:20036982 fatcat:6yozxoy4xne5pjgigzcmbqlfzi

Autonomous dynamics in neural networks: the dHAN concept and associative thought processes

Claudius Gros
2007 AIP Conference Proceedings  
The autonomous dynamics needs a long-term working-point optimization which acquires within the dHAN concept a dual functionality: It stabilizes the time development of the associative thought process and  ...  An associative thought-process is characterized, within this approach, by a time-series of transient attractors. Each transient state corresponds to a stored information, a memory.  ...  A short-term memory can speed-up the learning process substantially as it stabilizes external patterns and hence gives the system time to consolidate long-term synaptic plasticity. • Systems using sparse  ... 
doi:10.1063/1.2709594 fatcat:kkyw63csbvgdlfboudeqjziz54

Dynamic Adaptive Network Intelligence [article]

Richard Searle, Megan Bingham-Walker
2015 arXiv   pre-print
Accurate representational learning of both the explicit and implicit relationships within data is critical to the ability of machines to perform more complex and abstract reasoning tasks.  ...  We describe the efficient weakly supervised learning of such inferences by our Dynamic Adaptive Network Intelligence (DANI) model.  ...  Although we report the application of DANI as an independent framework for learning representation, we recognize that our system could be employed to condition the input and intermediate layers of neural  ... 
arXiv:1511.06379v1 fatcat:2obpgubyjnfqhl7hwpw2tfh6qy

Intrinsically Sparse Long Short-Term Memory Networks [article]

Shiwei Liu, Decebal Constantin Mocanu, Mykola Pechenizkiy
2019 arXiv   pre-print
Long Short-Term Memory (LSTM) has achieved state-of-the-art performances on a wide range of tasks.  ...  Its outstanding performance is guaranteed by the long-term memory ability which matches the sequential data perfectly and the gating structure controlling the information flow.  ...  Introduction In recent years, Long Short-Term Memory (LSTM) has returned to people's attention with its outstanding performance in speech recognition (Graves et al., 2013) , neural machine translation  ... 
arXiv:1901.09208v1 fatcat:auj2bmoajfdjxfsvrptrt4banq

Properties and mechanisms of olfactory learning and memory

Michelle T. Tong, Shane T. Peace, Thomas A. Cleland
2014 Frontiers in Behavioral Neuroscience  
We describe the properties of odor learning intrinsic to the olfactory bulb and review the utility of the olfactory system of adult rodents as a memory system in which to study the cellular mechanisms  ...  2014) Properties and mechanisms of olfactory learning and memory.  ...  STRUCTURAL MECHANISMS IN THE OB Long-term potentiation Long-term potentiation has been clearly if sparsely observed in the early olfactory system, notably within piriform cortex and its ascending synapses  ... 
doi:10.3389/fnbeh.2014.00238 pmid:25071492 pmcid:PMC4083347 fatcat:oeht2fvpzfbmbfvs3rtwv6smoi

A Vision Architecture [article]

Christoph von der Malsburg
2014 arXiv   pre-print
In this interpretation the permanent connectivity of cortex is an overlay of well-structured networks, nets, which are formed on the slow time-scale of learning by self-interaction of the network under  ...  We are offering a particular interpretation (well within the range of experimentally and theoretically accepted notions) of neural connectivity and dynamics and discuss it as the data-and-process architecture  ...  Visual input selects a sparse subset of units (vertical arrows). Neurons within units have WTA dynamics.  ... 
arXiv:1407.1642v1 fatcat:dcwkwdvg25hrdk2hancvupbov4

Learning precise spatiotemporal sequences via biophysically realistic circuits with modular structure [article]

Ian Cone, Harel Shouval
2020 bioRxiv   pre-print
The ability to express and learn temporal sequences is an essential part of learning and memory.  ...  This model provides a possible framework for biologically realistic sequence learning and memory, and is in agreement with recent experimental results, which have shown sequence dependent plasticity in  ...  Owing to these inputs and due to its strong recurrent connectivity, this network possesses a long-term memory of the state of the columnar network.  ... 
doi:10.1101/2020.04.17.046862 fatcat:uceqxeg43vduzmbzqjubx3tpki

Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation [article]

Tejas D. Kulkarni, Karthik R. Narasimhan, Ardavan Saeedi, Joshua B. Tenenbaum
2016 arXiv   pre-print
Learning goal-directed behavior in environments with sparse feedback is a major challenge for reinforcement learning algorithms.  ...  A top-level value function learns a policy over intrinsic goals, and a lower-level function learns a policy over atomic actions to satisfy the given goals. h-DQN allows for flexible goal specifications  ...  Hernandez-Gardiol and Mahadevan [19] combined hierarchical RL with a variable length short-term memory of high-level decisions.  ... 
arXiv:1604.06057v2 fatcat:p33suojusrcpfpg4ybc4hrfj6y

Brain Decoding from Functional MRI Using Long Short-Term Memory Recurrent Neural Networks [chapter]

Hongming Li, Yong Fan
2018 Lecture Notes in Computer Science  
In this study, we develop a deep learning based framework for brain decoding by leveraging recent advances in sequence modeling using long short-term memory (LSTM) recurrent neural networks (RNNs).  ...  , and LSTM RNNs are adopted to learn decoding mappings between functional profiles and brain states.  ...  On the other hand, recurrent neural networks (RNNs) with long short-term memory (LSTM) [10] have achieved remarkable advances in sequence modeling [11] , and these techniques might be powerful alternatives  ... 
doi:10.1007/978-3-030-00931-1_37 pmid:30320311 pmcid:PMC6180332 fatcat:w2f675q4jbd5tjwfxak2kihv3q

Off-line memory reprocessing following on-line unsupervised learning strongly improves recognition performance in a hierarchical visual memory

Jenia Jitsev, Christoph von der Malsburg
2010 The 2010 International Joint Conference on Neural Networks (IJCNN)  
Surprisingly, the positive effect turns out to be independent of synapse-specific plasticity, relying entirely on a homeostatic mechanism equalizing the intrinsic excitability levels of the units within  ...  Recently, experience-driven unsupervised learning was shown to create combinatorial parts-based representations in a model of hierarchical visual memory.  ...  During on-line learning, the bottomup and lateral synaptic structure within a lower memory layer is developed to represent local facial features and their relations.  ... 
doi:10.1109/ijcnn.2010.5596765 dblp:conf/ijcnn/JitsevM10 fatcat:ujacyp5v4zgzvaifoksixezudy

Maze Learning using a Hyperdimensional Predictive Processing Cognitive Architecture [article]

Alexander Ororbia, M. Alex Kelly
2022 arXiv   pre-print
We test CogNGen on four maze-learning tasks, including those that test memory and planning, and find that CogNGen matches performance of deep reinforcement learning models and exceeds on a task designed  ...  to test memory.  ...  Memory For CogNGen, we model both short and long-term memory using the MINERVA 2 model of human memory [9] .  ... 
arXiv:2204.00619v2 fatcat:l5zoat2inzcqhczr2qtmtmea6i

The less things change, the more they are different: contributions of long-term synaptic plasticity and homeostasis to memory

S. Schacher, J.-Y. Hu
2014 Learning & memory (Cold Spring Harbor, N.Y.)  
as a consequence of the accommodative interactions between long-term synaptic plasticity and homeostasis.  ...  An important cellular mechanism contributing to the strength and duration of memories is activity-dependent alterations in the strength of synaptic connections within the neural circuit encoding the memory  ...  Coexpression of homeostatic and plasticity mechanisms regulate basal synaptic transmission Reversal of short-and long-term plasticity or short-and long-term memory typically results in the reversal of  ... 
doi:10.1101/lm.027326.112 pmid:24532836 pmcid:PMC3929853 fatcat:cgpb322lcvbwbmgiisi75qa3pe

Multi-View Structural Local Subspace Tracking

Jie Guo, Tingfa Xu, Guokai Shi, Zhitao Rao, Xiangmin Li
2017 Sensors  
The proposed model not only exploits the intrinsic relationship among target candidates and their local patches, but also takes advantages of both sparse representation and incremental subspace learning  ...  Zhang et al. [33] proposed a structural sparse tracking algorithm which combines global and partial models together, then used the multi-task framework to exploit the intrinsic relationship among different  ...  We use it to account for the long-term memory of the target.  ... 
doi:10.3390/s17040666 pmid:28333088 pmcid:PMC5419779 fatcat:txgmsggatbfrla5jboby4ri4im
« Previous Showing results 1 — 15 out of 20,654 results