Filters








1,959 Hits in 2.4 sec

Modeling Compositionality with Multiplicative Recurrent Neural Networks [article]

Ozan İrsoy, Claire Cardie
2015 arXiv   pre-print
We present the multiplicative recurrent neural network as a general model for compositional meaning in language, and evaluate it on the task of fine-grained sentiment analysis.  ...  We establish a connection to the previously investigated matrix-space models for compositionality, and show they are special cases of the multiplicative recurrent net.  ...  On the other hand, recurrent neural networks (RNNs), a neural network architecture with sequential prediction capabilities, implicitly model compositionality when applied to natural language sentences.  ... 
arXiv:1412.6577v3 fatcat:76bthmmiojahfeg7nnogate6km

Representing Compositionality based on Multiple Timescales Gated Recurrent Neural Networks with Adaptive Temporal Hierarchy for Character-Level Language Models

Dennis Singh Moirangthem, Jegyung Son, Minho Lee
2017 Proceedings of the 2nd Workshop on Representation Learning for NLP  
The temporal hierarchy is introduced in the language model by utilizing a Gated Recurrent Neural Network with multiple timescales.  ...  The experiments show that the use of multiple timescales in a Neural Language Model (NLM) enables improved performance despite having fewer parameters and with no additional computation requirements.  ...  We propose a recurrent neural network based CLM with temporal hierarchies using a multilayer gated recurrent neural network.  ... 
doi:10.18653/v1/w17-2616 dblp:conf/rep4nlp/MoirangthemSL17 fatcat:okgbalqxtffezb45fdfhz5vl3m

Linguistic generalization and compositionality in modern artificial neural networks [article]

Marco Baroni
2019 arXiv   pre-print
In the last decade, deep artificial neural networks have achieved astounding performance in many natural language processing tasks.  ...  Given the high productivity of language, these models must possess effective generalization abilities.  ...  Supervised Sequence Labelling with Recurrent Neural Net- works. Springer, Berlin, 2012. [27] Tomas Mikolov. Statistical language models based on neural networks.  ... 
arXiv:1904.00157v2 fatcat:7ja6wz474vfmlllijyz5haby6e

Recognition of Visually Perceived Compositional Human Actions by Multiple Spatio-Temporal Scales Recurrent Neural Networks [article]

Haanvid Lee, Minju Jung, Jun Tani
2017 arXiv   pre-print
The proposed multiple spatio-temporal scales recurrent neural network (MSTRNN) model is derived by introducing multiple timescale recurrent dynamics to the conventional convolutional neural network model  ...  The current paper proposes a novel neural network model for recognizing visually perceived human actions.  ...  This leads to our novel proposal of multiple spatio-temporal scales recurrent neural network (MSTRNN) model in the current study. In the experiments, MSTRNNs were compared with MSTNNs and LRCNs.  ... 
arXiv:1602.01921v3 fatcat:sdz5w5djhnhsnixn5vl57oxb4a

Recurrent Convolutional Neural Networks for Discourse Compositionality [article]

Nal Kalchbrenner, Phil Blunsom
2013 arXiv   pre-print
The discourse model extends the sentence model and is based on a recurrent neural network that is conditioned in a novel way both on the current sentence and on the current speaker.  ...  The sentence model adopts convolution as the central operation for composing semantic vectors and is based on a novel hierarchical convolutional neural network.  ...  Recurrent Convolutional Neural Network The discourse model coupled to the sentence model is based on a RNN architecture with inputs from a HCNN and with the recurrent and output weights conditioned on  ... 
arXiv:1306.3584v1 fatcat:axym5i6aoje2bmgxhtbwed4574

Convolutional Neural Networks with Recurrent Neural Filters

Yi Yang
2018 Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing  
We introduce a class of convolutional neural networks (CNNs) that utilize recurrent neural networks (RNNs) as convolution filters.  ...  In this work, we model convolution filters with RNNs that naturally capture compositionality and long-term dependencies in language.  ...  Our recurrent neural filters (RNFs) can naturally deal with language compositionality with a recurrent function that models word relations, and they are also able to implicitly model long-term dependencies  ... 
doi:10.18653/v1/d18-1109 dblp:conf/emnlp/Yang18 fatcat:r6cbi5bmzncinmkppqrz5qt5xy

Convolutional Neural Networks with Recurrent Neural Filters [article]

Yi Yang
2018 arXiv   pre-print
We introduce a class of convolutional neural networks (CNNs) that utilize recurrent neural networks (RNNs) as convolution filters.  ...  In this work, we model convolution filters with RNNs that naturally capture compositionality and long-term dependencies in language.  ...  Our recurrent neural filters (RNFs) can naturally deal with language compositionality with a recurrent function that models word relations, and they are also able to implicitly model long-term dependencies  ... 
arXiv:1808.09315v1 fatcat:dir4qt6v5rdmbpkjyrmojrebnq

Deep Recursive Neural Networks for Compositionality in Language

Ozan Irsoy, Claire Cardie
2014 Neural Information Processing Systems  
deep recurrent neural networks.  ...  In this work we introduce a new architecture -a deep recursive neural network (deep RNN)constructed by stacking multiple recursive layers.  ...  In fact, a recurrent neural network is simply a recursive neural network with a particular structure (see Figure 1c ).  ... 
dblp:conf/nips/IrsoyC14 fatcat:3xkl7nzzmnco3aqipz4zg5kcmi

Restricted Recurrent Neural Tensor Networks: Exploiting Word Frequency and Compositionality [article]

Alexandre Salle, Aline Villavicencio
2018 arXiv   pre-print
Recurrent neural tensor networks (RNTN) increase capacity using distinct hidden layer weights for each word, but with greater costs in memory usage.  ...  Increasing the capacity of recurrent neural networks (RNN) usually involves augmenting the size of the hidden layer, with significant increase of computational cost.  ...  Sutskever et al. (2011) increased the performance of a character-level language model with a multiplicative RNN (m-RNN), the factored approximation of a recurrent neural tensor network (RNTN), which maps  ... 
arXiv:1704.00774v3 fatcat:2cecnnd2hze4neqfr5k5ppjimq

Multiple Time Scales Recurrent Neural Network for Complex Action Acquisition

Cangelosi Angelo
2011 Frontiers in Computational Neuroscience  
recurrent neural network expert systems [5] .  ...  Yamashita and Tani [3] were inspired by the latest biological observations of the brain to develop a completely new model of action sequence learning known as Multiple Timescales Recurrent Neural Network  ... 
doi:10.3389/conf.fncom.2011.52.00009 fatcat:2yut65cdqfclbh5qrys4f26koa

Do Character-Level Neural Network Language Models Capture Knowledge of Multiword Expression Compositionality?

Ali Hakimi Parizi, Paul Cook
2018 Linguistic Annotation Workshop  
In this paper we propose the first model for multiword expression (MWE) compositionality prediction based on character-level neural network language models.  ...  Experimental results on two kinds of MWEs (noun compounds and verb-particle constructions) and two languages (English and German) suggest that character-level neural network language models capture knowledge  ...  We train character-level language models based on recurrent neural networks -including long short-term memory (LSTM, Hochreiter and Schmidhuber, 1997) and gated recurrent unit (GRU, Cho et al., 2014  ... 
dblp:conf/acllaw/PariziC18 fatcat:356gj3tz6reuhl6ao43a55fvtm

DAG-Structured Long Short-Term Memory for Semantic Compositionality

Xiaodan Zhu, Parinaz Sobhani, Hongyu Guo
2016 Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies  
Recurrent neural networks, particularly long short-term memory (LSTM), have recently shown to be very effective in a wide range of sequence modeling problems, core to which is effective learning of distributed  ...  From a more general viewpoint, the proposed models incorporate additional prior knowledge into recurrent neural networks, which is interesting to us, considering most NLP tasks have relatively small training  ...  The proposed models unify the compositional power of recurrent neural networks (RNN) and additional prior knowledge.  ... 
doi:10.18653/v1/n16-1106 dblp:conf/naacl/ZhuSG16 fatcat:owgilhx2dbfjldwjlavqkkyysu

Compositionality for Recursive Neural Networks [article]

Martha Lewis
2019 arXiv   pre-print
This mapping suggests a number of lines of research for both categorical compositional vector space models of meaning and for recursive neural network models of compositionality.  ...  In this paper I show how a linear simplification of recursive neural tensor network models can be mapped directly onto the categorical approach, giving a way of computing the required matrices and tensors  ...  neural network such as long short-term memory networks or gated recurrent units.  ... 
arXiv:1901.10723v1 fatcat:jnubsz2ntbbktbghyhpvpund7q

Towards Abstraction from Extraction: Multiple Timescale Gated Recurrent Unit for Summarization

Minsoo Kim, Dennis Singh Moirangthem, Minho Lee
2016 Proceedings of the 1st Workshop on Representation Learning for NLP  
The proposed Multiple Timescale model of the Gated Recurrent Unit (MT-GRU) is implemented in the encoderdecoder setting to better deal with the presence of multiple compositionalities in larger texts.  ...  The results also show that the temporal hierarchies help improve the ability of seq2seq models to capture compositionalities better without the presence of highly complex architectural hierarchies.  ...  Previous works such as (Yamashita and Tani, 2008)'s Multiple Timescale Recurrent Neural Network (MTRNN) have employed temporal hierarchy in motion prediction.  ... 
doi:10.18653/v1/w16-1608 dblp:conf/rep4nlp/KimML16 fatcat:vigsvuixzfhk3meovbfvk6snbi

Towards Abstraction from Extraction: Multiple Timescale Gated Recurrent Unit for Summarization [article]

Minsoo Kim, Moirangthem Dennis Singh, Minho Lee
2016 arXiv   pre-print
The proposed Multiple Timescale model of the Gated Recurrent Unit (MTGRU) is implemented in the encoder-decoder setting to better deal with the presence of multiple compositionalities in larger texts.  ...  The results also show that the temporal hierarchies help improve the ability of seq2seq models to capture compositionalities better without the presence of highly complex architectural hierarchies.  ...  Previous works such as (Yamashita and Tani, 2008)'s Multiple Timescale Recurrent Neural Network (MTRNN) have employed temporal hierarchy in motion prediction.  ... 
arXiv:1607.00718v1 fatcat:fmc3rz3sgzbf7eaohxxix3swha
« Previous Showing results 1 — 15 out of 1,959 results