A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf
.
Filters
Learning Sentence Embeddings for Coherence Modelling and Beyond
[article]
2019
arXiv
pre-print
In this work, we show that a new type of sentence embedding learned through self-supervision can be applied effectively to text coherence tasks while serving as a window through which deeper understanding ...
We present a novel and effective technique for performing text coherence tasks while facilitating deeper insights into the data. ...
NSERC invests annually over $1 billion in people, discovery and innovation. ...
arXiv:1804.08053v2
fatcat:s7zltj7offcwpozpvaqzpuqera
Learning and Evaluating Sparse Interpretable Sentence Embeddings
[article]
2018
arXiv
pre-print
We further introduce a novel, quantitative and automated evaluation metric for sentence embedding interpretability, based on topic coherence methods. ...
Previous research on word embeddings has shown that sparse representations, which can be either learned on top of existing dense embeddings or obtained through model constraints during training time, have ...
An interpretable sentence representation has further applications beyond model understanding: for example, it allows us to develop a sentence similarity measure, that can justify why two sentences are ...
arXiv:1809.08621v2
fatcat:newrr5qp3zbclg3jkobhjyqexm
Learning and Evaluating Sparse Interpretable Sentence Embeddings
2018
Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP
We further introduce a novel, quantitative and automated evaluation metric for sentence embedding interpretability, based on topic coherence methods. ...
Previous research on word embeddings has shown that sparse representations, which can be either learned on top of existing dense embeddings or obtained through model constraints during training time, have ...
An interpretable sentence representation has further applications beyond model understanding: for example, it allows us to develop a sentence similarity measure, that can justify why two sentences are ...
doi:10.18653/v1/w18-5422
dblp:conf/emnlp/TrifonovGPH18
fatcat:d5sws3e26jhzllgqb33hlgucau
Feature Weight Tuning for Recursive Neural Networks
[article]
2014
arXiv
pre-print
The proposed model can be viewed as incorporating a more powerful compositional function for embedding acquisition in recursive neural networks. ...
This paper addresses how a recursive neural network model can automatically leave out useless information and emphasize important evidence, in other words, to perform "weight tuning" for higher-level representation ...
Recursive and recurrent [19, 20] models constitute two types of commonly used frameworks for sentence-level embedding acquisition. ...
arXiv:1412.3714v2
fatcat:fdrdvqpnhvdm7mysbiwubg2muy
Sentence-Level Evidence Embedding for Claim Verification with Hierarchical Attention Networks
2019
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
Our model consists of three main components: 1) A coherence-based attention layer embeds coherent evidence considering the claim and sentences from relevant articles; 2) An entailment-based attention layer ...
attends on sentences that can semantically infer the claim on top of the first attention; and 3) An output layer predicts the verdict based on the embedded evidence. ...
Acknowledgment This work was partly supported by Hong Kong RGC GRF (14232816, 14209416, 14204118), NSFC (61877020) and SCSE-SUG grant M4082038 at NTU. ...
doi:10.18653/v1/p19-1244
dblp:conf/acl/MaGJW19
fatcat:eaj2eia44ngavhk62bbgodo2v4
Language modeling via stochastic processes
[article]
2022
arXiv
pre-print
Compared to domain-specific methods and fine-tuning GPT2 across a variety of text domains, TC improves performance on text infilling and discourse coherence. ...
Using this representation, the language model can generate text by first implicitly generating a document plan via a stochastic process, and then generating text that is consistent with this latent plan ...
The authors would give special thanks to CoColab members Mike Wu, Gabriel Poesia, Ali Malik and Alex Tamkin for their support and incredibly helpful discussions. ...
arXiv:2203.11370v1
fatcat:kodw6rv3snarbhl4dtfzmgoslm
Narrative Incoherence Detection
[article]
2021
arXiv
pre-print
Despite its simple setup, this task is challenging as the model needs to understand and analyze a multi-sentence narrative, and predict incoherence at the sentence level. ...
We observe that while token-level modeling has better performance when the input contains fewer sentences, sentence-level modeling performs better on longer narratives and possesses an advantage in efficiency ...
Despite its importance, learning inter-sentential coherence remains an open challenge, as it requires (i) understanding, extracting, and representing the high-level semantic flow for a given text; (ii) ...
arXiv:2012.11157v2
fatcat:skrfyuplurfj5morvv5swjojg4
Toward Better Storylines with Sentence-Level Language Models
[article]
2020
arXiv
pre-print
Since it does not need to model fluency, the sentence-level language model can focus on longer range dependencies, which are crucial for multi-sentence coherence. ...
Rather than dealing with individual words, our method treats the story so far as a list of pre-trained sentence embeddings and predicts an embedding for the next sentence, which is more efficient than ...
Government is authorized to reproduce and distribute reprints for governmental purposes notwithstanding any copyright annotation therein. ...
arXiv:2005.05255v1
fatcat:wbosyghiqzatjgn2wfrszj4m5a
Do We Need Neural Models to Explain Human Judgments of Acceptability?
[article]
2019
arXiv
pre-print
We test the ability of computational language models, simple language features, and word embeddings to predict native English speakers judgments of acceptability on English-language essays written by non-native ...
We find that much of the sentence acceptability variance can be captured by a combination of features including misspellings, word order, and word similarity (Pearson's r = 0.494). ...
Neural language models have shown evidence of acquiring deeper grammatical competence beyond mere n-gram statistics (Gulordava et al., 2018) , suggesting that the models are a good basis for modelling ...
arXiv:1909.08663v2
fatcat:anwgnde5rjbpbizgaxos4gwasu
Discourse-Based Objectives for Fast Unsupervised Sentence Representation Learning
[article]
2017
arXiv
pre-print
This work presents a novel objective function for the unsupervised training of neural network sentence encoders. ...
It exploits signals from paragraph-level discourse coherence to train these models to understand text. ...
Looking beyond work on unsupervised pretraining: Li and Hovy (2014) and Li and Jurafsky (2016) use representation learning systems to directly model the problem of sentence order recovery, but focus ...
arXiv:1705.00557v1
fatcat:cch2bg2o45dehosi5n4ffj7tdi
Jointly Modeling Embedding and Translation to Bridge Video and Language
[article]
2015
arXiv
pre-print
model for exploring the relationships between visual content and sentence semantics. ...
Our proposed LSTM-E consists of three components: a 2-D and/or 3-D deep convolutional neural networks for learning powerful video representation, a deep RNN for generating sentences, and a joint embedding ...
Joint Modeling Embedding and Translation Following the relevance and coherence criteria, this work proposes a Long Short-Term Memory with visualsemantic Embedding (LSTM-E) model for video description generation ...
arXiv:1505.01861v3
fatcat:tvpbppgjabaytl2xlsi2mrtxmi
Jointly Modeling Embedding and Translation to Bridge Video and Language
2016
2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
This paper presents a novel unified framework, named Long Short-Term Memory with visual-semantic Embedding (LSTM-E), which can simultaneously explore the learning of LSTM and visual-semantic embedding. ...
The former aims to locally maximize the probability of generating the next word given previous words and visual content, while the latter is to create a visual-semantic embedding space for enforcing the ...
Joint Modeling Embedding and Translation Following the relevance and coherence criteria, this work proposes a Long Short-Term Memory with visualsemantic Embedding (LSTM-E) model for video description generation ...
doi:10.1109/cvpr.2016.497
dblp:conf/cvpr/PanMYLR16
fatcat:i3zdepzpsvfqhco67ulg5hycaq
Phrase-BERT: Improved Phrase Embeddings from BERT with an Application to Corpus Exploration
[article]
2021
arXiv
pre-print
Crowdsourced evaluations demonstrate that this phrase-based topic model produces more coherent and meaningful topics than baseline word and phrase-level topic models, further validating the utility of ...
Finally, as a case study, we show that Phrase-BERT embeddings can be easily integrated with a simple autoencoder to build a phrase-based neural topic model that interprets topics as mixtures of words and ...
We would also like to thank He He for recommending the use of PAWS as evaluation data, Tu Vu for the valuable suggestions on paper writing, and the anonymous reviewers for their insightful feedback. ...
arXiv:2109.06304v2
fatcat:b4wgaaxb6fdcbocwkacyy4xqlu
Enhancing Pointer Network for Sentence Ordering with Pairwise Ordering Predictions
2020
PROCEEDINGS OF THE THIRTIETH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE AND THE TWENTY-EIGHTH INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE
We also evaluate our sentence ordering models on a downstream task, multi-document summarization, and the summaries reordered by our model achieve the best coherence scores. ...
with respect to the candidate sentence, and the HISTORY module measures the local coherence between several (e.g., 2) previously ordered sentences and the candidate sentence, without the influence of ...
Acknowledgments The authors were supported by Beijing Advanced Innovation Center for Language Resources, National Natural Sci- ...
doi:10.1609/aaai.v34i05.6492
fatcat:ctysouwm2rb33id5f6a6ornfnu
Learning to Extract Coherent Summary via Deep Reinforcement Learning
[article]
2018
arXiv
pre-print
The RNES model learns to optimize coherence and informative importance of the summary simultaneously. ...
The proposed neural coherence model obviates the need for feature engineering and can be trained in an end-to-end fashion using unlabeled data. ...
Since REINFORCE algorithm converges very slowly, we pretrain the RNES model with supervised learning. The neural coherence model is also trained and then fixed for the coherence scoring. ...
arXiv:1804.07036v1
fatcat:hobbf7sibjbd7mcsn2nzmqdwve
« Previous
Showing results 1 — 15 out of 32,556 results