Filters








32,556 Hits in 4.5 sec

Learning Sentence Embeddings for Coherence Modelling and Beyond [article]

Tanner Bohn, Yining Hu, Jinhang Zhang, Charles X. Ling
2019 arXiv   pre-print
In this work, we show that a new type of sentence embedding learned through self-supervision can be applied effectively to text coherence tasks while serving as a window through which deeper understanding  ...  We present a novel and effective technique for performing text coherence tasks while facilitating deeper insights into the data.  ...  NSERC invests annually over $1 billion in people, discovery and innovation.  ... 
arXiv:1804.08053v2 fatcat:s7zltj7offcwpozpvaqzpuqera

Learning and Evaluating Sparse Interpretable Sentence Embeddings [article]

Valentin Trifonov, Octavian-Eugen Ganea, Anna Potapenko, Thomas Hofmann
2018 arXiv   pre-print
We further introduce a novel, quantitative and automated evaluation metric for sentence embedding interpretability, based on topic coherence methods.  ...  Previous research on word embeddings has shown that sparse representations, which can be either learned on top of existing dense embeddings or obtained through model constraints during training time, have  ...  An interpretable sentence representation has further applications beyond model understanding: for example, it allows us to develop a sentence similarity measure, that can justify why two sentences are  ... 
arXiv:1809.08621v2 fatcat:newrr5qp3zbclg3jkobhjyqexm

Learning and Evaluating Sparse Interpretable Sentence Embeddings

Valentin Trifonov, Octavian-Eugen Ganea, Anna Potapenko, Thomas Hofmann
2018 Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP  
We further introduce a novel, quantitative and automated evaluation metric for sentence embedding interpretability, based on topic coherence methods.  ...  Previous research on word embeddings has shown that sparse representations, which can be either learned on top of existing dense embeddings or obtained through model constraints during training time, have  ...  An interpretable sentence representation has further applications beyond model understanding: for example, it allows us to develop a sentence similarity measure, that can justify why two sentences are  ... 
doi:10.18653/v1/w18-5422 dblp:conf/emnlp/TrifonovGPH18 fatcat:d5sws3e26jhzllgqb33hlgucau

Feature Weight Tuning for Recursive Neural Networks [article]

Jiwei Li
2014 arXiv   pre-print
The proposed model can be viewed as incorporating a more powerful compositional function for embedding acquisition in recursive neural networks.  ...  This paper addresses how a recursive neural network model can automatically leave out useless information and emphasize important evidence, in other words, to perform "weight tuning" for higher-level representation  ...  Recursive and recurrent [19, 20] models constitute two types of commonly used frameworks for sentence-level embedding acquisition.  ... 
arXiv:1412.3714v2 fatcat:fdrdvqpnhvdm7mysbiwubg2muy

Sentence-Level Evidence Embedding for Claim Verification with Hierarchical Attention Networks

Jing Ma, Wei Gao, Shafiq Joty, Kam-Fai Wong
2019 Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics  
Our model consists of three main components: 1) A coherence-based attention layer embeds coherent evidence considering the claim and sentences from relevant articles; 2) An entailment-based attention layer  ...  attends on sentences that can semantically infer the claim on top of the first attention; and 3) An output layer predicts the verdict based on the embedded evidence.  ...  Acknowledgment This work was partly supported by Hong Kong RGC GRF (14232816, 14209416, 14204118), NSFC (61877020) and SCSE-SUG grant M4082038 at NTU.  ... 
doi:10.18653/v1/p19-1244 dblp:conf/acl/MaGJW19 fatcat:eaj2eia44ngavhk62bbgodo2v4

Language modeling via stochastic processes [article]

Rose E Wang, Esin Durmus, Noah Goodman, Tatsunori Hashimoto
2022 arXiv   pre-print
Compared to domain-specific methods and fine-tuning GPT2 across a variety of text domains, TC improves performance on text infilling and discourse coherence.  ...  Using this representation, the language model can generate text by first implicitly generating a document plan via a stochastic process, and then generating text that is consistent with this latent plan  ...  The authors would give special thanks to CoColab members Mike Wu, Gabriel Poesia, Ali Malik and Alex Tamkin for their support and incredibly helpful discussions.  ... 
arXiv:2203.11370v1 fatcat:kodw6rv3snarbhl4dtfzmgoslm

Narrative Incoherence Detection [article]

Deng Cai and Yizhe Zhang and Yichen Huang and Wai Lam and Bill Dolan
2021 arXiv   pre-print
Despite its simple setup, this task is challenging as the model needs to understand and analyze a multi-sentence narrative, and predict incoherence at the sentence level.  ...  We observe that while token-level modeling has better performance when the input contains fewer sentences, sentence-level modeling performs better on longer narratives and possesses an advantage in efficiency  ...  Despite its importance, learning inter-sentential coherence remains an open challenge, as it requires (i) understanding, extracting, and representing the high-level semantic flow for a given text; (ii)  ... 
arXiv:2012.11157v2 fatcat:skrfyuplurfj5morvv5swjojg4

Toward Better Storylines with Sentence-Level Language Models [article]

Daphne Ippolito, David Grangier, Douglas Eck, Chris Callison-Burch
2020 arXiv   pre-print
Since it does not need to model fluency, the sentence-level language model can focus on longer range dependencies, which are crucial for multi-sentence coherence.  ...  Rather than dealing with individual words, our method treats the story so far as a list of pre-trained sentence embeddings and predicts an embedding for the next sentence, which is more efficient than  ...  Government is authorized to reproduce and distribute reprints for governmental purposes notwithstanding any copyright annotation therein.  ... 
arXiv:2005.05255v1 fatcat:wbosyghiqzatjgn2wfrszj4m5a

Do We Need Neural Models to Explain Human Judgments of Acceptability? [article]

Wang Jing, M. A. Kelly (The Pennsylvania State University), David Reitter
2019 arXiv   pre-print
We test the ability of computational language models, simple language features, and word embeddings to predict native English speakers judgments of acceptability on English-language essays written by non-native  ...  We find that much of the sentence acceptability variance can be captured by a combination of features including misspellings, word order, and word similarity (Pearson's r = 0.494).  ...  Neural language models have shown evidence of acquiring deeper grammatical competence beyond mere n-gram statistics (Gulordava et al., 2018) , suggesting that the models are a good basis for modelling  ... 
arXiv:1909.08663v2 fatcat:anwgnde5rjbpbizgaxos4gwasu

Discourse-Based Objectives for Fast Unsupervised Sentence Representation Learning [article]

Yacine Jernite, Samuel R. Bowman, David Sontag
2017 arXiv   pre-print
This work presents a novel objective function for the unsupervised training of neural network sentence encoders.  ...  It exploits signals from paragraph-level discourse coherence to train these models to understand text.  ...  Looking beyond work on unsupervised pretraining: Li and Hovy (2014) and Li and Jurafsky (2016) use representation learning systems to directly model the problem of sentence order recovery, but focus  ... 
arXiv:1705.00557v1 fatcat:cch2bg2o45dehosi5n4ffj7tdi

Jointly Modeling Embedding and Translation to Bridge Video and Language [article]

Yingwei Pan, Tao Mei, Ting Yao, Houqiang Li, Yong Rui
2015 arXiv   pre-print
model for exploring the relationships between visual content and sentence semantics.  ...  Our proposed LSTM-E consists of three components: a 2-D and/or 3-D deep convolutional neural networks for learning powerful video representation, a deep RNN for generating sentences, and a joint embedding  ...  Joint Modeling Embedding and Translation Following the relevance and coherence criteria, this work proposes a Long Short-Term Memory with visualsemantic Embedding (LSTM-E) model for video description generation  ... 
arXiv:1505.01861v3 fatcat:tvpbppgjabaytl2xlsi2mrtxmi

Jointly Modeling Embedding and Translation to Bridge Video and Language

Yingwei Pan, Tao Mei, Ting Yao, Houqiang Li, Yong Rui
2016 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)  
This paper presents a novel unified framework, named Long Short-Term Memory with visual-semantic Embedding (LSTM-E), which can simultaneously explore the learning of LSTM and visual-semantic embedding.  ...  The former aims to locally maximize the probability of generating the next word given previous words and visual content, while the latter is to create a visual-semantic embedding space for enforcing the  ...  Joint Modeling Embedding and Translation Following the relevance and coherence criteria, this work proposes a Long Short-Term Memory with visualsemantic Embedding (LSTM-E) model for video description generation  ... 
doi:10.1109/cvpr.2016.497 dblp:conf/cvpr/PanMYLR16 fatcat:i3zdepzpsvfqhco67ulg5hycaq

Phrase-BERT: Improved Phrase Embeddings from BERT with an Application to Corpus Exploration [article]

Shufan Wang and Laure Thompson and Mohit Iyyer
2021 arXiv   pre-print
Crowdsourced evaluations demonstrate that this phrase-based topic model produces more coherent and meaningful topics than baseline word and phrase-level topic models, further validating the utility of  ...  Finally, as a case study, we show that Phrase-BERT embeddings can be easily integrated with a simple autoencoder to build a phrase-based neural topic model that interprets topics as mixtures of words and  ...  We would also like to thank He He for recommending the use of PAWS as evaluation data, Tu Vu for the valuable suggestions on paper writing, and the anonymous reviewers for their insightful feedback.  ... 
arXiv:2109.06304v2 fatcat:b4wgaaxb6fdcbocwkacyy4xqlu

Enhancing Pointer Network for Sentence Ordering with Pairwise Ordering Predictions

Yongjing Yin, Fandong Meng, Jinsong Su, Yubin Ge, Lingeng Song, Jie Zhou, Jiebo Luo
2020 PROCEEDINGS OF THE THIRTIETH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE AND THE TWENTY-EIGHTH INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE  
We also evaluate our sentence ordering models on a downstream task, multi-document summarization, and the summaries reordered by our model achieve the best coherence scores.  ...  with respect to the candidate sentence, and the HISTORY module measures the local coherence between several (e.g., 2) previously ordered sentences and the candidate sentence, without the influence of  ...  Acknowledgments The authors were supported by Beijing Advanced Innovation Center for Language Resources, National Natural Sci-  ... 
doi:10.1609/aaai.v34i05.6492 fatcat:ctysouwm2rb33id5f6a6ornfnu

Learning to Extract Coherent Summary via Deep Reinforcement Learning [article]

Yuxiang Wu, Baotian Hu
2018 arXiv   pre-print
The RNES model learns to optimize coherence and informative importance of the summary simultaneously.  ...  The proposed neural coherence model obviates the need for feature engineering and can be trained in an end-to-end fashion using unlabeled data.  ...  Since REINFORCE algorithm converges very slowly, we pretrain the RNES model with supervised learning. The neural coherence model is also trained and then fixed for the coherence scoring.  ... 
arXiv:1804.07036v1 fatcat:hobbf7sibjbd7mcsn2nzmqdwve
« Previous Showing results 1 — 15 out of 32,556 results