Filters








2,461 Hits in 6.2 sec

Discourse-Aware Neural Rewards for Coherent Text Generation [article]

Antoine Bosselut, Asli Celikyilmaz, Xiaodong He, Jianfeng Gao, Po-Sen Huang, Yejin Choi
2018 arXiv   pre-print
In this paper, we investigate the use of discourse-aware rewards with reinforcement learning to guide a model to generate long, coherent text.  ...  In particular, we propose to learn neural rewards to model cross-sentence ordering as a means to approximate desired discourse structure.  ...  Acknowledgments We thank Chloe Kiddon for helpful discussions in the early stages of this work.  ... 
arXiv:1805.03766v1 fatcat:wgo5gzkvqnha3mjdecmedgfzlm

Discourse-Aware Neural Rewards for Coherent Text Generation

Antoine Bosselut, Asli Celikyilmaz, Xiaodong He, Jianfeng Gao, Po-Sen Huang, Yejin Choi
2018 Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)  
In this paper, we investigate the use of discourse-aware rewards with reinforcement learning to guide a model to generate long, coherent text.  ...  In particular, we propose to learn neural rewards to model cross-sentence ordering as a means to approximate desired discourse structure.  ...  In this paper, we investigate learning neural rewards and their use in a reinforcement learning regime with a specific focus on learning more discourse-aware and coherent text generation.  ... 
doi:10.18653/v1/n18-1016 dblp:conf/naacl/BosselutCHGHC18 fatcat:lnlx5v3eojhn7frb5szkzutela

Modeling Coherence for Discourse Neural Machine Translation [article]

Hao Xiong, Zhongjun He, Hua Wu, Haifeng Wang
2018 arXiv   pre-print
Moreover, our model generates more discourse coherent text and obtains +2.2 BLEU improvements when evaluated by discourse metrics.  ...  Next, we deliberate the preliminary produced translations, and train the model to learn the policy that produces discourse coherent text by a reward teacher.  ...  Our Approach Intuitively, it is plausible that a model with external context summarized from entire text, trained by a discourse-aware reward can generate more accurate and discourse coherent translations  ... 
arXiv:1811.05683v1 fatcat:jlkhpmb3qjdptdkvpeqkw467cm

Knowledge-aware Document Summarization: A Survey of Knowledge, Embedding Methods and Architectures [article]

Yutong Qu, Wei Emma Zhang, Jian Yang, Lingfei Wu, Jia Wu
2022 arXiv   pre-print
Previous works reported that knowledge-embedded document summarizers excel at generating superior digests, especially in terms of informativeness, coherence, and fact consistency.  ...  Knowledge-aware methods have boosted a range of natural language processing applications over the last decades.  ...  Document Summarization (DS) aims to generate an abridged version of single or multiple topic-related texts as concise and coherent as possible while preserving the salient and factually consistent information  ... 
arXiv:2204.11190v2 fatcat:rx7x6l47xzaa3gubjfvyaexpem

Abstractive and mixed summarization for long-single documents [article]

Roger Barrull, Jugal Kalita
2020 arXiv   pre-print
The lack of diversity in the datasets available for automatic summarization of documents has meant that the vast majority of neural models for automatic summarization have been trained with news articles  ...  Discourse-aware attention model The discourse-aware attention model can be seen as the adaptation of the PGN with coverage model for summarizing long-structured documents, e.g., scientific papers.  ...  Bhm (Bhm 2019) highlighted the limitations of ROUGEbased rewarders and proposed neural network-based rewarders to predict the similarity between document and summary.  ... 
arXiv:2007.01918v1 fatcat:s3adavfxmrbkjhndrpsyp6tcre

Automatic Story Generation: Challenges and Attempts [article]

Amal Alabdulkarim, Siyan Li, Xiangyu Peng
2021 arXiv   pre-print
Shed light on emerging and often overlooked challenges such as creativity and discourse.  ...  The scope of this survey paper is to explore the challenges in automatic story generation. We hope to contribute in the following ways: 1.  ...  In order to incorporates paragraph-level information to generate coherent commonsense inferences from narratives, Gabriel et al. (2020) proposed a discourse-aware model PARA-COMeT.  ... 
arXiv:2102.12634v1 fatcat:b67pi4zy5fc4dp4edidecwo54a

An Entity-Driven Framework for Abstractive Summarization [article]

Eva Sharma, Luyang Huang, Zhe Hu, Lu Wang
2019 arXiv   pre-print
compression and abstraction to generate the final summary, which is trained with rewards to promote coherence, conciseness, and clarity.  ...  In this paper, we introduce SENECA, a novel System for ENtity-drivEn Coherent Abstractive summarization framework that leverages entity information to generate informative and coherent abstracts.  ...  We also thank Sebastian Sehrmann for sharing their outputs on NYT and CNN/DM and Asli Celikyilmaz for sharing their summaries on CNN/DM.  ... 
arXiv:1909.02059v1 fatcat:lqs5onifobd27m2svcuet2jloq

Corpora for Document-Level Neural Machine Translation

Siyou Liu, Xiaojun Zhang
2020 International Conference on Language Resources and Evaluation  
In recent years, there have been more interests in modelling larger context for the state-of-the-art neural machine translation (NMT).  ...  Instead of translating sentences in isolation, document-level machine translation aims to capture discourse dependencies across sentences by considering a document as a whole.  ...  Besides, Xiong et al. (2019) proposed to use discourse context and reward to refine the translation quality from the perspective of coherence.  ... 
dblp:conf/lrec/LiuZ20 fatcat:36nsgsllajd7pjzctz3pd5l6ni

Leveraging Discourse Rewards for Document-Level Neural Machine Translation [article]

Inigo Jauregi Unanue, Nazanin Esmaili, Gholamreza Haffari, Massimo Piccardi
2020 arXiv   pre-print
Therefore, in this paper we propose a training approach that explicitly optimizes two established discourse metrics, lexical cohesion (LC) and coherence (COH), by using a reinforcement learning objective  ...  However, document-level translation models are usually not trained to explicitly ensure discourse quality.  ...  Sameen Maruf for her feedback on an early version of this paper.  ... 
arXiv:2010.03732v2 fatcat:lwlvl7tpxvaajoanrxoxozq7ze

BIGPATENT: A Large-Scale Dataset for Abstractive and Coherent Summarization

Eva Sharma, Chen Li, Lu Wang
2019 Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics  
Most existing text summarization datasets are compiled from the news domain, where summaries have a flattened discourse structure.  ...  Finally, we train and evaluate baselines and popular learning models on BIGPATENT to shed light on new challenges and motivate future directions for summarization research. 1 BIGPATENT dataset is available  ...  We also thank the anonymous reviewers for their constructive suggestions.  ... 
doi:10.18653/v1/p19-1212 dblp:conf/acl/SharmaLW19 fatcat:hidyvt4wg5hvpb2xiahu2jhgma

DiscoDVT: Generating Long Text with Discourse-Aware Discrete Variational Transformer [article]

Haozhe Ji, Minlie Huang
2021 arXiv   pre-print
Despite the recent advances in applying pre-trained language models to generate high-quality texts, generating long passages that maintain long-range coherence is yet challenging for these models.  ...  long texts with better long-range coherence.  ...  We preserve a maximum number of 64 subwords for the input prompt and 512 subwords for the story, respectively. A.3.2 Details on Discourse Annotations Preparation  ... 
arXiv:2110.05999v1 fatcat:wy3vgkgo3rgoto5fnwvtuk5ar4

Evaluation Benchmarks and Learning Criteria for Discourse-Aware Sentence Representations [article]

Mingda Chen, Zewei Chu, Kevin Gimpel
2019 arXiv   pre-print
We also propose a variety of training objectives that makes use of natural annotations from Wikipedia to build sentence encoders capable of modeling discourse.  ...  Acknowledgments We thank Jonathan Kummerfeld for helpful discussions about the IRC Disentanglement dataset, Davis Yoshida for discussions about BERT, and the anonymous reviewers for their feedback that  ...  Discourse-aware neural rewards for coherent text generation.  ... 
arXiv:1909.00142v2 fatcat:dylxptj4avgunlrmy337ztkrsu

DAGN: Discourse-Aware Graph Network for Logical Reasoning [article]

Yinya Huang, Meng Fang, Yu Cao, Liwei Wang, Xiaodan Liang
2021 arXiv   pre-print
We propose a discourse-aware graph network (DAGN) that reasons relying on the discourse structure of the texts.  ...  The model encodes discourse information as a graph with elementary discourse units (EDUs) and discourse relations, and learns the discourse-aware features via a graph network for downstream QA tasks.  ...  Acknowledgements The authors would like to thank Wenge Liu, Jianheng Tang, Guanlin Li and Wei Wang for their support and useful discussions.  ... 
arXiv:2103.14349v2 fatcat:p7stettavzazjbwtvr3nwpmsru

A Survey on Document-level Neural Machine Translation: Methods and Evaluation [article]

Sameen Maruf, Fahimeh Saleh, Gholamreza Haffari
2021 arXiv   pre-print
In addition, we cover evaluation strategies that have been introduced to account for the improvements in document MT, including automatic metrics and discourse-targeted test sets.  ...  With the resurgence of neural networks, the translation quality surpasses that of the translations obtained using statistical techniques for most language-pairs.  ...  [36] developed a discourse-aware evaluation metric by first generating discourse trees for the translation output and reference using a discourse parser (lexicalised and un-lexicalised), and then computing  ... 
arXiv:1912.08494v3 fatcat:x7kcds5hxnabzj5c2ilw6xyqiu

Evaluation Benchmarks and Learning Criteria for Discourse-Aware Sentence Representations

Mingda Chen, Zewei Chu, Kevin Gimpel
2019 Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)  
Prior work on pretrained sentence embeddings and benchmarks focuses on the capabilities of representations for stand-alone sentences.  ...  We propose DiscoEval, a test suite of tasks to evaluate whether sentence representations include information about the role of a sentence in its discourse context.  ...  Acknowledgments We thank Jonathan Kummerfeld for helpful discussions about the IRC Disentanglement dataset, Davis Yoshida for discussions about BERT, and the anonymous reviewers for their feedback that  ... 
doi:10.18653/v1/d19-1060 dblp:conf/emnlp/ChenCG19 fatcat:tlpzcde6unfflo6ymuwrtbhjkq
« Previous Showing results 1 — 15 out of 2,461 results