Filters








2,180 Hits in 10.9 sec

Modeling Compositionality with Multiplicative Recurrent Neural Networks [article]

Ozan İrsoy, Claire Cardie
2015 arXiv   pre-print
We present the multiplicative recurrent neural network as a general model for compositional meaning in language, and evaluate it on the task of fine-grained sentiment analysis.  ...  We establish a connection to the previously investigated matrix-space models for compositionality, and show they are special cases of the multiplicative recurrent net.  ...  Intuitively, a matrix embedding of a word is desired in order to capture operator semantics: the embedding should model how a word transforms meaning when it is applied to a context.  ... 
arXiv:1412.6577v3 fatcat:76bthmmiojahfeg7nnogate6km

Compositional Semantics [chapter]

Zhiyuan Liu, Yankai Lin, Maosong Sun
2020 Representation Learning for Natural Language Processing  
Therefore, compositional semantics has remained a core task in NLP.  ...  In this chapter, we first introduce various models for binary semantic composition, including additive models and multiplicative models.  ...  Reference [15] proposes a recursive matrix-vector model (MV-RNN) which captures constituent parsing tree structure information by assigning a matrix-vector representation for each constituent.  ... 
doi:10.1007/978-981-15-5573-2_3 fatcat:uu524rdsxnd7flgrvprsuxaicq

Learning Phrase Embeddings from Paraphrases with GRUs [article]

Zhihao Zhou, Lifu Huang, Heng Ji
2017 arXiv   pre-print
In this work, we propose to take advantage of large-scaled paraphrase database and present a pair-wise gated recurrent units (pairwise-GRU) framework to generate compositional phrase representations.  ...  Previous studies either learn non-compositional phrase representations with general word embedding learning techniques or learn compositional phrase representations based on syntactic structures, which  ...  The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the U.S. Government.  ... 
arXiv:1710.05094v1 fatcat:klt2imth65fw3aftacwsjtmq24

Gradual Learning of Matrix-Space Models of Language for Sentiment Analysis

Shima Asaadi, Sebastian Rudolph
2017 Proceedings of the 2nd Workshop on Representation Learning for NLP  
In this work, we investigate the problem of learning matrix representations of words. We present a learning approach for compositional matrix-space models for the task of sentiment analysis.  ...  Beyond the popular vector space models, matrix representations for words have been proposed, since then, matrix multiplication can serve as natural composition operation.  ...  As an early work in compositional distributional semantics, Mitchell and Lapata (2010) propose vector composition models with additive and multiplicative functions as the composition operations in semantic  ... 
doi:10.18653/v1/w17-2621 dblp:conf/rep4nlp/AsaadiR17 fatcat:sivmavepmvdhtdlffb2ya3jrcu

Compositional matrix-space models of language: Definitions, properties, and learning methods

Shima Asaadi, Eugenie Giesbrecht, Sebastian Rudolph
2021 Natural Language Engineering  
We give an in-depth account of compositional matrix-space models (CMSMs), a type of generic models for natural language, wherein compositionality is realized via matrix multiplication.  ...  We argue for the structural plausibility of this model and show that it is able to cover and combine various common compositional natural language processing approaches.  ...  partially supported by the German Research Foundation (DFG) within the Research Training Group QuantLA (GRK 1763) and by the Federal Ministry of Education and Research of Germany BMBF through the Center for  ... 
doi:10.1017/s1351324921000206 fatcat:jt24zv5bfvgdnmt2mpitptewym

A Re-ranking Model for Dependency Parser with Recursive Convolutional Neural Network [article]

Chenxi Zhu, Xipeng Qiu, Xinchi Chen, Xuanjing Huang
2015 arXiv   pre-print
We propose a recursive convolutional neural network (RCNN) architecture to capture syntactic and compositional-semantic representations of phrases and words in a dependency tree.  ...  In this work, we address the problem to model all the nodes (words or phrases) in a dependency tree with the dense representations.  ...  Acknowledgments We would like to thank the anonymous reviewers for their valuable comments.  ... 
arXiv:1505.05667v1 fatcat:dk5ixndtmzezdgn6as5zo23ofm

A Re-ranking Model for Dependency Parser with Recursive Convolutional Neural Network

Chenxi Zhu, Xipeng Qiu, Xinchi Chen, Xuanjing Huang
2015 Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)  
We propose a recursive convolutional neural network (RCNN) architecture to capture syntactic and compositional-semantic representations of phrases and words in a dependency tree.  ...  In this work, we address the problem to model all the nodes (words or phrases) in a dependency tree with the dense representations.  ...  Acknowledgments We would like to thank the anonymous reviewers for their valuable comments.  ... 
doi:10.3115/v1/p15-1112 dblp:conf/acl/ZhuQCH15 fatcat:yu7fxkek25fy5ddsahvazrc6xy

Self-Adaptive Hierarchical Sentence Model [article]

Han Zhao, Zhengdong Lu, Pascal Poupart
2015 arXiv   pre-print
The ability to accurately model a sentence at varying stages (e.g., word-phrase-sentence) plays a central role in natural language processing.  ...  AdaSent effectively forms a hierarchy of representations from words to phrases and then to sentences through recursive gated local composition of adjacent segments.  ...  Han Zhao thanks Tao Cai and Baotian Hu at Noah's Ark Lab for their technical support and helpful discussions. This work is supported in part by China National 973 project 2014CB340301.  ... 
arXiv:1504.05070v2 fatcat:ozp6au5bzrbgrkeslo4pu4snly

Predicting Polarities of Tweets by Composing Word Embeddings with Long Short-Term Memory

Xin Wang, Yuanchao Liu, Chengjie SUN, Baoxun Wang, Xiaolong Wang
2015 Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)  
In this paper, we introduce Long Short-Term Memory (LSTM) recurrent network for twitter sentiment prediction.  ...  With the help of gates and constant error carousels in the memory block structure, the model could handle interactions between words through a flexible compositional function.  ...  Acknowledgments We thank Deyuan Zhang, Lei Cui, Feng Liu and Ming Liu for their great help. We thank the anonymous reviewers for their insightful feedbacks on this work.  ... 
doi:10.3115/v1/p15-1130 dblp:conf/acl/WangLSWW15 fatcat:5p5uazlhsfcsvhiocqvhwwpmau

Document Modeling with Gated Recurrent Neural Network for Sentiment Classification

Duyu Tang, Bing Qin, Ting Liu
2015 Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing  
To address this, we introduce a neural network model to learn vector-based document representation in a unified, bottom-up fashion.  ...  Document level sentiment classification remains a challenge: encoding the intrinsic relations between sentences in the semantic meaning of a document.  ...  Acknowledgments The authors give great thanks to Yaming Sun and Jiwei Li for the fruitful discussions.  ... 
doi:10.18653/v1/d15-1167 dblp:conf/emnlp/TangQL15 fatcat:aw6bp24ofbfgpbesi7fsnhrbki

When Are Tree Structures Necessary for Deep Learning of Representations? [article]

Jiwei Li, Minh-Thang Luong, Dan Jurafsky, Eudard Hovy
2015 arXiv   pre-print
We then introduce a method for allowing recurrent models to achieve similar performance: breaking long sentences into clause-like units at punctuation and processing them separately before combining.  ...  In this paper we benchmark recursive neural models against sequential recurrent neural models (simple recurrent and LSTM models), enforcing apples-to-apples comparison as much as possible.  ...  Acknowledgments We would especially like to thank Richard Socher and Kai-Sheng Tai for insightful comments, advice, and suggestions.  ... 
arXiv:1503.00185v5 fatcat:iay2f5nrurgmhonxp6l2bexchy

An Attention-Based Hybrid Neural Network for Document Modeling

Dengchao HE, Hongjun ZHANG, Wenning HAO, Rui ZHANG, Huan HAO
2017 IEICE transactions on information and systems  
Concretely, our model adopts a bidirectional LSTM module with word-level attention to extract semantic information for each sentence in text and subsequently learns high level features via a dynamic convolution  ...  In this paper, proposed is a novel attention-based hybrid neural network model, which would extract semantic features of text hierarchically.  ...  The significant challenge for modeling documents is to capture semantic features and to perform compositions over variable-length documents.  ... 
doi:10.1587/transinf.2016edl8231 fatcat:yzzq2plf35hrdgrqux2j5cqx3y

When Are Tree Structures Necessary for Deep Learning of Representations?

Jiwei Li, Thang Luong, Dan Jurafsky, Eduard Hovy
2015 Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing  
We then introduce a method for allowing recurrent models to achieve similar performance: breaking long sentences into clause-like units at punctuation and processing them separately before combining.  ...  In this paper, we benchmark recursive neural models against sequential recurrent neural models, enforcing applesto-apples comparison as much as possible.  ...  Acknowledgments We would especially like to thank Richard Socher and Kai-Sheng Tai for insightful comments, advice, and suggestions.  ... 
doi:10.18653/v1/d15-1278 dblp:conf/emnlp/LiLJH15 fatcat:bcvvcfcvenbubl6qdaz66bua2q

Deep Learning applied to NLP [article]

Marc Moreno Lopez, Jugal Kalita
2017 arXiv   pre-print
CNNs are responsible for major breakthroughs in Image Classification and are the core of most Computer Vision systems today.  ...  More recently CNNs have been applied to problems in Natural Language Processing and gotten some interesting results.  ...  by embedding them in a low dimensional vector space, while preserving distinctions of word and sentence order crucial for capturing nuanced semantics.  ... 
arXiv:1703.03091v1 fatcat:3grekqst4jbr3np5yv4vzqa4ze

Model-Free Context-Aware Word Composition

Bo An, Xianpei Han, Le Sun
2018 International Conference on Computational Linguistics  
Word composition is a promising technique for representation learning of large linguistic units (e.g., phrases, sentences and documents).  ...  To address this issue, we propose a model-free context-aware word composition model, which employs the latent semantic information as global context for learning representations.  ...  Moreover, we sincerely thank the reviewers for their valuable comments.  ... 
dblp:conf/coling/AnHS18 fatcat:wwhwsn6j6zd4dpl46ibg4aeh3i
« Previous Showing results 1 — 15 out of 2,180 results