Filters








6,094 Hits in 2.8 sec

Closing Brackets with Recurrent Neural Networks

Natalia Skachkova, Thomas Trost, Dietrich Klakow
2018 Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP  
Currently, recurrent neural networks (RNNs) are extensively used for this task.  ...  We investigate whether they are capable of learning the rules of opening and closing brackets by applying them to synthetic Dyck languages that consist of different types of brackets.  ...  Neural Network Architecture We use three different RNN architectures for our experiments: Elman-RNN (abbreviated as SRNN for simple RNN), GRU (gated recurrent unit), and LSTM (long short-term memory).  ... 
doi:10.18653/v1/w18-5425 dblp:conf/emnlp/SkachkovaTK18 fatcat:2p2vs2bpmzbcpasdwamtyr35zu

Visualisation and 'Diagnostic Classifiers' Reveal how Recurrent and Recursive Neural Networks Process Hierarchical Structure (Extended Abstract)

Dieuwke Hupkes, Willem Zuidema
2018 Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence  
In this paper, we investigate how recurrent neural networks can learn and process languages with hierarchical, compositional semantics.  ...  We find that simple recurrent networks cannot find a generalising solution to this task, but gated recurrent neural networks perform surprisingly well: networks learn to predict the outcome of the arithmetic  ...  next number should be added or subtracted when a bracket closes.  ... 
doi:10.24963/ijcai.2018/796 dblp:conf/ijcai/HupkesZ18 fatcat:zwhzmwx4pbb4pipj2utwum3dsq

Evaluating the Ability of LSTMs to Learn Context-Free Grammars

Luzi Sennhauser, Robert Berwick
2018 Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP  
We explore this question with a well-formed bracket prediction task using two types of brackets modeled by an LSTM.  ...  While long short-term memory (LSTM) neural net architectures are designed to capture sequence information, human language is generally composed of hierarchical structures.  ...  Long Short-Term Memory Long-Short-Term-Memory networks (LSTM) (Hochreiter and Schmidhuber, 1997 ) are a variant of recurrent neural networks (RNNs).  ... 
doi:10.18653/v1/w18-5414 dblp:conf/emnlp/SennhauserB18 fatcat:jmwqh5kxxrflplf24psmha4qja

Visualisation and 'diagnostic classifiers' reveal how recurrent and recursive neural networks process hierarchical structure [article]

Dieuwke Hupkes, Sara Veldhoen, Willem Zuidema
2018 arXiv   pre-print
We investigate how neural networks can learn and process languages with hierarchical, compositional semantics.  ...  As a next step, we investigate recurrent neural networks, and show that a gated recurrent unit, that processes its input incrementally, also performs very well on this task.  ...  subtracted when a bracket closes (in Figure 2b captured by the variable mode).  ... 
arXiv:1711.10203v2 fatcat:6qdv7jvhsrdttffwrssowyp46e

Minimum Description Length Recurrent Neural Networks [article]

Nur Lan, Michal Geyer, Emmanuel Chemla, Roni Katzir
2022 arXiv   pre-print
We train neural networks to optimize a Minimum Description Length score, i.e., to balance between the complexity of the network and its accuracy at a task.  ...  Moreover, they often do so with 100 transparent. We thus provide formal proofs that their perfect accuracy holds not only on a given test set, but for any input sequence.  ...  Each task was run three times with different random seeds. 2 To compare the performance of the MDLoptimized recurrent neural networks (MDLRNNs) with classical models, we trained standard RNNs on the same  ... 
arXiv:2111.00600v4 fatcat:mg6lcj47yrgidmbur33tfym4qi

Visualisation and 'Diagnostic Classifiers' Reveal How Recurrent and Recursive Neural Networks Process Hierarchical Structure

Dieuwke Hupkes, Sara Veldhoen, Willem Zuidema
2018 The Journal of Artificial Intelligence Research  
We investigate how neural networks can learn and process languages with hierarchical, compositional semantics.  ...  As a next step, we investigate recurrent neural networks, and show that a gated recurrent unit, that processes its input incrementally, also performs very well on this task: the network learns to predict  ...  subtracted when a bracket closes (in Figure 2b captured by the variable mode).  ... 
doi:10.1613/jair.1.11196 fatcat:g647zy5ksnfyvhibn7zwbegxnq

Explainable artificial intelligence for mechanics: physics-informing neural networks for constitutive models [article]

Arnd Koeppe and Franz Bamer and Michael Selzer and Britta Nestler and Bernd Markert
2021 arXiv   pre-print
(Artificial) neural networks have become increasingly popular in mechanics to accelerate computations with model order reduction techniques and as universal models for a wide variety of materials.  ...  In mechanics, the new and active field of physics-informed neural networks attempts to mitigate this disadvantage by designing deep neural networks on the basis of mechanical knowledge.  ...  Ministry of Education and Research (BMBF) in the project FestBatt (project number 03XP0174E) and by the Ministry of Science, Research and Art Baden-Württemberg in the project MoMaF -Science Data Center, with  ... 
arXiv:2104.10683v4 fatcat:wwujekwjqfe2dht46s2pg5bjr4

Evaluating the Ability of LSTMs to Learn Context-Free Grammars [article]

Luzi Sennhauser, Robert C. Berwick
2018 arXiv   pre-print
We explore this question with a well-formed bracket prediction task using two types of brackets modeled by an LSTM.  ...  While long short-term memory (LSTM) neural net architectures are designed to capture sequence information, human language is generally composed of hierarchical structures.  ...  Long Short-Term Memory Long-Short-Term-Memory networks (LSTM) (Hochreiter and Schmidhuber, 1997 ) are a variant of recurrent neural networks (RNNs).  ... 
arXiv:1811.02611v1 fatcat:q7dwxdl7fbdrnfm3z5h6yoey3i

Span-Based Constituency Parsing with a Structure-Label System and Provably Optimal Dynamic Oracles

James Cross, Liang Huang
2016 Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing  
Parsing accuracy using efficient greedy transition systems has improved dramatically in recent years thanks to neural networks.  ...  Training with this oracle, we achieve the best F 1 scores on both English and French of any parser that does not use reranking or external data.  ...  Our neural model processes the sentence once for each parse with a recurrent network.  ... 
doi:10.18653/v1/d16-1001 dblp:conf/emnlp/CrossH16 fatcat:zc2w5kuxgvbobp7mt5po3fmzxu

Span-Based Constituency Parsing with a Structure-Label System and Provably Optimal Dynamic Oracles [article]

James Cross, Liang Huang
2016 arXiv   pre-print
Parsing accuracy using efficient greedy transition systems has improved dramatically in recent years thanks to neural networks.  ...  Training with this oracle, we achieve the best F1 scores on both English and French of any parser that does not use reranking or external data.  ...  Our neural model processes the sentence once for each parse with a recurrent network.  ... 
arXiv:1612.06475v1 fatcat:vnlv3hencvdhpj6h5ey43g4whe

Tackling Graphical Natural Language Processing's Problems with Recurrent Neural Networks

Ali Sami Sosa, Saja Majeed Mohammed, Haider Hadi Abbas, Israa Al Barazanchi
2019 Journal of Southwest Jiaotong University  
graph recurrent neural network.  ...  the Local using recurrent neural network.  ...  Encoding with Graph Recurrent Neural Networks Since being introduced, the graph recurrent neural networks (GRN's), as mentioned by the author in an article [8] , have long been neglected.  ... 
doi:10.35741/issn.0258-2724.54.5.35 fatcat:6vimnp7scrahhbfl2cdmwa3q5u

On the Ability and Limitations of Transformers to Recognize Formal Languages [article]

Satwik Bhattamishra, Kabir Ahuja, Navin Goyal
2020 arXiv   pre-print
Past works suggest that LSTMs generalize very well on regular languages and have close connections with counter languages.  ...  Transformers have supplanted recurrent models in a large number of NLP tasks. However, the differences in their abilities to model different syntactic properties remain largely unknown.  ...  Closing brackets with recurrent neu- ral networks. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpret- ing Neural Networks for NLP, pages 232-239, Brus- sels, Belgium.  ... 
arXiv:2009.11264v2 fatcat:bdyjkxoyyvfyzo5afdasl7ehnm

Explainable Artificial Intelligence for Mechanics: Physics-Explaining Neural Networks for Constitutive Models

Arnd Koeppe, Franz Bamer, Michael Selzer, Britta Nestler, Bernd Markert
2022 Frontiers in Materials  
(Artificial) neural networks have become increasingly popular in mechanics and materials sciences to accelerate computations with model order reduction techniques and as universal models for a wide variety  ...  The new and active field of physics-informed neural networks attempts to mitigate this disadvantage by designing deep neural networks on the basis of mechanical knowledge.  ...  behavior and to explain the resulting recurrent neural networks' behavior.  ... 
doi:10.3389/fmats.2021.824958 fatcat:vu2vli4q3rbtxacxlrs25ztq44

Explainable Artificial Intelligence for Mechanics: Physics-Explaining Neural Networks for Constitutive Models

Arnd Koeppe, Franz Bamer, Michael Selzer, Britta Nestler, Bernd Markert
2022 Frontiers in Materials 8  
Artificial) neural networks have become increasingly popular in mechanics and materials sciences to accelerate computations with model order reduction techniques and as universal models for a wide variety  ...  The new and active field of physics-informed neural networks attempts to mitigate this disadvantage by designing deep neural networks on the basis of mechanical knowledge.  ...  behavior and to explain the resulting recurrent neural networks' behavior.  ... 
doi:10.18154/rwth-2022-01227 fatcat:nqskhaxzhjejbbynm2yusc3kri

Study and Overview on System Feedback Representations in Control Modeling with Artificial Neural Networks (ANN) Platform

Sayanti Roy, Zinkar Das, Subham Ghosh, Biswarup Neogi
2013 International Journal of Computer Applications  
A brief literature review on control theory approach with feedback block modeling and signal flow graphs along with the contribution of artificial neural networks in the field of control system is also  ...  The implementation of any block diagram with artificial neural networks using MATLAB (R2008b) simulator and the neural modelof the training state and the performance obtained from the graphs is the basic  ...  Thus it plays a vital role in the study of a special class of neural networks known as the recurrent networks.  ... 
doi:10.5120/12125-8228 fatcat:4nkvyenofjc5xehvssbzia7r7a
« Previous Showing results 1 — 15 out of 6,094 results