Filters








286,697 Hits in 6.7 sec

ASHuR: Evaluation of the Relation Summary-Content Without Human Reference Using ROUGE

Alan Ramírez-Noriega, Reyes Juárez-Ramírez, Samantha Jiménez, Sergio Inzunza, Yobani Martínez-Ramírez
<span title="">2018</span> <i title="AEPress, s.r.o."> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/rsgu6oqy7fcfdl4hs633t7yefy" style="color: black;">Computing and informatics</a> </i> &nbsp;
ROUGE metrics can automatically evaluate a summary, but it needs a model summary built by humans.  ...  The goal of this study is to find a quantitative relation between an article content and its summary using ROUGE tests without a model summary built by humans.  ...  The purpose of this article is to evaluate a summary without the human model input.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.4149/cai_2018_2_509">doi:10.4149/cai_2018_2_509</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/rrmiwye6mnexhfo53nwh2abslm">fatcat:rrmiwye6mnexhfo53nwh2abslm</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20210814170131/http://www.cai.sk/ojs/index.php/cai/article/download/2018_2_509/895" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/64/eb/64eb42739b79c98ab6d962c0175583adb81a5f51.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.4149/cai_2018_2_509"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> Publisher / doi.org </button> </a>

QARLA

Enrique Amigó, Julio Gonzalo, Anselmo Peñas, Felisa Verdejo
<span title="">2005</span> <i title="Association for Computational Linguistics"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/5n6volmnonf5tn6xputi5f2t3e" style="color: black;">Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics - ACL &#39;05</a> </i> &nbsp;
The input of the framework is a set of manual (reference) summaries, a set of baseline (automatic) summaries and a set of similarity metrics between summaries.  ...  Compared to previous approaches, our framework is able to combine different metrics and evaluate the quality of a set of metrics without any a-priori weighting of their relative importance.  ...  , the automatic evaluation of systems consists of computing some similarity metric between the system output and a human model summary.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.3115/1219840.1219875">doi:10.3115/1219840.1219875</a> <a target="_blank" rel="external noopener" href="https://dblp.org/rec/conf/acl/AmigoGPV05.html">dblp:conf/acl/AmigoGPV05</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/nvrylvil45dl3a6rjk3gnrzyhu">fatcat:nvrylvil45dl3a6rjk3gnrzyhu</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20060613212237/http://acl.ldc.upenn.edu:80/P/P05/P05-1035.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/27/cd/27cd6626af76117e04d4ba2631a2dccf642307f5.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.3115/1219840.1219875"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> Publisher / doi.org </button> </a>

The challenging task of summary evaluation: an overview

Elena Lloret, Laura Plaza, Ahmet Aker
<span title="2017-09-02">2017</span> <i title="Springer Nature America, Inc"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/qiptgj2ubngu3hrrsrkbdvpchi" style="color: black;">Language Resources and Evaluation</a> </i> &nbsp;
evaluation of summaries.  ...  To perform an adequate evaluation is of great relevance to ensure that automatic summaries can be useful for the context and/or application they are generated for.  ...  Recently, Cabrera-Diego et al. (2016) proposed a trivergent model to evaluate summaries without human references.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1007/s10579-017-9399-2">doi:10.1007/s10579-017-9399-2</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/tduxzlv2hfbfvd6evzoxn5xibu">fatcat:tduxzlv2hfbfvd6evzoxn5xibu</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20180722145507/https://rua.ua.es/dspace/bitstream/10045/71549/2/2017_Lloret_etal_LangResources&amp;Evaluation_preprint.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/33/53/3353d2823b0071519350e235cee4be756eba651e.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1007/s10579-017-9399-2"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> springer.com </button> </a>

Estimating Summary Quality with Pairwise Preferences

Markus Zopf
<span title="">2018</span> <i title="Association for Computational Linguistics"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/d5ex6ucxtrfz3clshlkh3f6w2q" style="color: black;">Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)</a> </i> &nbsp;
Automatic evaluation systems in the field of automatic summarization have been relying on the availability of gold standard summaries for over ten years.  ...  In this paper, we propose an alternative evaluation approach based on pairwise preferences of sentences. In comparison to gold standard summaries, they are simpler and cheaper to obtain.  ...  Model-free Evaluation Model-free evaluation methods Jensen-Shannon divergence (Louis and Nenkova, 2013) do not require human input such as gold standard summaries and can therefore be applied without  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.18653/v1/n18-1152">doi:10.18653/v1/n18-1152</a> <a target="_blank" rel="external noopener" href="https://dblp.org/rec/conf/naacl/Zopf18.html">dblp:conf/naacl/Zopf18</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/2xrsv3z3xfcxtiznj7r3g2isgq">fatcat:2xrsv3z3xfcxtiznj7r3g2isgq</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20200307123051/https://www.aclweb.org/anthology/N18-1152.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/b7/ee/b7eeacd13a1083565183d2330c6d12f70cbc6975.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.18653/v1/n18-1152"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> Publisher / doi.org </button> </a>

Older versions of the ROUGEeval summarization evaluation system were easier to fool

Jonas Sjöbergh
<span title="">2007</span> <i title="Elsevier BV"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/v5dch4enzne6phusiwdh25za24" style="color: black;">Information Processing &amp; Management</a> </i> &nbsp;
We show some limitations of the ROUGE evaluation method for automatic summarization. We present a method for automatic summarization based on a Markov model of the source text.  ...  By a simple greedy word selection strategy, summaries with high ROUGE-scores are generated. These summaries would however not be considered good by human readers.  ...  Adapting the summary to the evaluation procedure gives large effects. Favoring short words is for instance bad when evaluating without stop words.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1016/j.ipm.2007.01.014">doi:10.1016/j.ipm.2007.01.014</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/nayyj6nnwbhbfmt3khqhir6njy">fatcat:nayyj6nnwbhbfmt3khqhir6njy</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20170809034935/http://dr-hato.se/research/abuserouge.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/09/ed/09ed404c1bca408a17c285992911c41261cb0082.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1016/j.ipm.2007.01.014"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> elsevier.com </button> </a>

Automatically Assessing Machine Summary Content Without a Gold Standard

Annie Louis, Ani Nenkova
<span title="">2013</span> <i title="MIT Press - Journals"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/e4yflo6sufcufjwloald7xudm4" style="color: black;">Computational Linguistics</a> </i> &nbsp;
The third technique improves standard automatic evaluations by expanding the set of available model summaries with chosen system summaries.  ...  We also explore ways of increasing evaluation quality when only one human model summary is available as a gold standard.  ...  Acknowledgments We would like to thank John Conroy for providing us with the summaries from the CLASSY system and Xi Lin for the implementation of the LSA-based summarizer.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1162/coli_a_00123">doi:10.1162/coli_a_00123</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/uxmusgylzfe7lovcfps5qjcogi">fatcat:uxmusgylzfe7lovcfps5qjcogi</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20130711075023/http://aclweb.org:80/anthology/J/J13/J13-2002.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/c9/33/c933abeae17d22c64cd4612f90f3078ab9f21b03.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1162/coli_a_00123"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="unlock alternate icon" style="background-color: #fb971f;"></i> mitpressjournals.org </button> </a>

Task Proposal: The TL;DR Challenge

Shahbaz Syed, Michael Völske, Martin Potthast, Nedim Lipka, Benno Stein, Hinrich Schütze
<span title="">2018</span> <i title="Association for Computational Linguistics"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/3rq6eimo35hlhbg4sx6i3ty2uq" style="color: black;">Proceedings of the 11th International Conference on Natural Language Generation</a> </i> &nbsp;
the door to all kinds of generative models.  ...  The challenge owes its name to the frequent practice of social media users to supplement long posts with a "TL;DR"for "too long; didn't read"-followed by a short summary as a courtesy to those who would  ...  By emphasizing on the effectiveness and limitations of exisiting models through our challenge, the NLG community can focus on novel models and evaluation measures for developing better summarization technology  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.18653/v1/w18-6538">doi:10.18653/v1/w18-6538</a> <a target="_blank" rel="external noopener" href="https://dblp.org/rec/conf/inlg/SyedVPLSS18.html">dblp:conf/inlg/SyedVPLSS18</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/hni36qatlve4jb7zoztttwcbu4">fatcat:hni36qatlve4jb7zoztttwcbu4</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20200509082706/https://www.aclweb.org/anthology/W18-6538.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/bc/5a/bc5aef23631ca78dcb8b32b1a619f8e7f0d35ef8.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.18653/v1/w18-6538"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> Publisher / doi.org </button> </a>

Sentence Mover's Similarity: Automatic Evaluation for Multi-Sentence Texts

Elizabeth Clark, Asli Celikyilmaz, Noah A. Smith
<span title="">2019</span> <i title="Association for Computational Linguistics"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/5n6volmnonf5tn6xputi5f2t3e" style="color: black;">Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics</a> </i> &nbsp;
We also show that sentence mover's similarity can be used as a reward when learning a generation model via reinforcement learning; we present both automatic and human evaluations of summaries learned in  ...  For evaluating machine-generated texts, automatic methods hold the promise of avoiding collection of human judgments, which can be expensive and time-consuming.  ...  automatic metrics and human evaluations.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.18653/v1/p19-1264">doi:10.18653/v1/p19-1264</a> <a target="_blank" rel="external noopener" href="https://dblp.org/rec/conf/acl/ClarkCS19.html">dblp:conf/acl/ClarkCS19</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/l2yirncppjax5fr2mslacwpf2e">fatcat:l2yirncppjax5fr2mslacwpf2e</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20200505220145/https://www.aclweb.org/anthology/P19-1264.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/6a/b3/6ab3f185d9e4f0c8cfa0331810e15f5db09035c3.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.18653/v1/p19-1264"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> Publisher / doi.org </button> </a>

A boundary-based tokenization technique for extractive text summarization

Nnaemeka M Oparauwah, Juliet N Odii, Ikechukwu I Ayogu, Vitalis C Iwuchukwu
<span title="2021-08-30">2021</span> <i title="GSC Online Press"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/pl6yccvwzfezniw5ildoicobw4" style="color: black;">World Journal of Advanced Research and Reviews</a> </i> &nbsp;
The need to extract and manage vital information contained in copious volumes of text documents has given birth to several automatic text summarization (ATS) approaches.  ...  An extractive summarization algorithm was further developed based on the proposed boundary-based tokenization method, as well as word length consideration to control redundancy in summary output.  ...  In recent studies, human evaluators have been shown to be effective in summary evaluation.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.30574/wjarr.2021.11.2.0351">doi:10.30574/wjarr.2021.11.2.0351</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/t67arfwm3neyzkai7sevgm63uy">fatcat:t67arfwm3neyzkai7sevgm63uy</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20210901050406/https://wjarr.com/sites/default/files/WJARR-2021-0351.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/81/60/81601e2c856dbb155b06879bb13cd61ae794718b.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.30574/wjarr.2021.11.2.0351"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> Publisher / doi.org </button> </a>

A compositional context sensitive multi-document summarizer

Ani Nenkova, Lucy Vanderwende, Kathleen McKeown
<span title="">2006</span> <i title="ACM Press"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/ibcfmixrofb3piydwg5wvir3t4" style="color: black;">Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval - SIGIR &#39;06</a> </i> &nbsp;
We carry out our analysis using datasets from the Document Understanding Conferences, studying not only the impact of these features on automatic summarizers, but also their role in human summarization  ...  The usual approach for automatic summarization is sentence extraction, where key sentences from the input documents are selected based on a suite of features.  ...  overlap with multiple human summaries/models. 1.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1145/1148170.1148269">doi:10.1145/1148170.1148269</a> <a target="_blank" rel="external noopener" href="https://dblp.org/rec/conf/sigir/NenkovaVM06.html">dblp:conf/sigir/NenkovaVM06</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/367skrssyjckjjgeyady7cofl4">fatcat:367skrssyjckjjgeyady7cofl4</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20080424162546/http://www.cis.upenn.edu/~nenkova/papers/fp285-nenkova.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/6f/2e/6f2ea95fa94284444684e8d6f1af62d0bc1cd14d.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1145/1148170.1148269"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> acm.org </button> </a>

Page 21 of Computational Linguistics Vol. 34, Issue 1 [page]

<span title="">2008</span> <i title="MIT Press Journals"> <a target="_blank" rel="noopener" href="https://archive.org/details/pub_computational-linguistics" style="color: black;">Computational Linguistics </a> </i> &nbsp;
Our evaluation was based on materials from the Document Understanding Con- ference (DUC 2003), which include multi-document summaries produced by human writers and by automatic summarization systems.  ...  Barzilay and Lapata Modeling Local Coherence 5.1 Modeling Summary coherence rating can be also formulated as a ranking learning task.  ... 
<span class="external-identifiers"> </span>
<a target="_blank" rel="noopener" href="https://archive.org/details/sim_computational-linguistics_2008-03_34_1/page/21" title="read fulltext microfilm" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Archive [Microfilm] <div class="menu fulltext-thumbnail"> <img src="https://archive.org/serve/sim_computational-linguistics_2008-03_34_1/__ia_thumb.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a>

GEMS: Generative Modeling for Evaluation of Summaries [chapter]

Rahul Katragadda
<span title="">2010</span> <i title="Springer Berlin Heidelberg"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/2w3awgokqne6te4nvlofavy5a4" style="color: black;">Lecture Notes in Computer Science</a> </i> &nbsp;
In this paper we present a Generative Modeling framework for evaluation of content of summaries.  ...  Automated evaluation is crucial in the context of automated text summaries, as is the case with evaluation of any of the language technologies.  ...  Intrinsic evaluations have taken two major forms: manual, in which one or more people evaluate the system produced summary and automatic, in which the summary is evaluated without the human in the loop  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1007/978-3-642-12116-6_61">doi:10.1007/978-3-642-12116-6_61</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/uxcxii5cr5cd3ai7waojkzwgme">fatcat:uxcxii5cr5cd3ai7waojkzwgme</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20170808211355/http://web2py.iiit.ac.in/publications/default/download/inproceedings.pdf.830e5ba20fa1028c.47454d532d4b6174726167616464612e706466.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/b3/44/b34462574c6b73712ebb1e306638b3bb70035781.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1007/978-3-642-12116-6_61"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> springer.com </button> </a>

BottleSum: Unsupervised and Self-supervised Sentence Summarization using the Information Bottleneck Principle

Peter West, Ari Holtzman, Jan Buys, Yejin Choi
<span title="">2019</span> <i title="Association for Computational Linguistics"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/u3ideoxy4fghvbsstiknuweth4" style="color: black;">Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)</a> </i> &nbsp;
In addition, we find that our selfsupervised abstractive model outperforms unsupervised baselines (including our own) by human evaluation along multiple attributes.  ...  Empirical results demonstrate that our extractive method outperforms other unsupervised models on multiple automatic metrics.  ...  We use the ABS model of Rush et al. (2015) as a baseline for automatic and human evaluation.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.18653/v1/d19-1389">doi:10.18653/v1/d19-1389</a> <a target="_blank" rel="external noopener" href="https://dblp.org/rec/conf/emnlp/WestHBC19.html">dblp:conf/emnlp/WestHBC19</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/a7rllnns5nhpbk7d2wtpgdp7ua">fatcat:a7rllnns5nhpbk7d2wtpgdp7ua</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20200822164858/https://www.aclweb.org/anthology/D19-1389.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/92/c2/92c21e244798f1f54fd02ff2cabfa4daced32d11.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.18653/v1/d19-1389"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> Publisher / doi.org </button> </a>

Modeling local coherence

Regina Barzilay, Mirella Lapata
<span title="">2005</span> <i title="Association for Computational Linguistics"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/5n6volmnonf5tn6xputi5f2t3e" style="color: black;">Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics - ACL &#39;05</a> </i> &nbsp;
Our experiments demonstrate that the induced model achieves significantly higher accuracy than a state-of-the-art coherence model.  ...  This paper considers the problem of automatic assessment of local coherence.  ...  Second, we compare the rankings produced by the model against human coherence judgments elicited for automatically generated summaries.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.3115/1219840.1219858">doi:10.3115/1219840.1219858</a> <a target="_blank" rel="external noopener" href="https://dblp.org/rec/conf/acl/BarzilayL05.html">dblp:conf/acl/BarzilayL05</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/7s4b2fye5vc4piwb5jh4r62lam">fatcat:7s4b2fye5vc4piwb5jh4r62lam</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20170810163425/http://ssli.ee.washington.edu/conferences/ACL2005/ACL/pdf/ACL18.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/9d/74/9d74eca00d4d0c4aa7c7369ae37d67498b37bf2f.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.3115/1219840.1219858"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> Publisher / doi.org </button> </a>

SummEval: Re-evaluating Summarization Evaluation

Alexander R. Fabbri, Wojciech Kryściński, Bryan McCann, Caiming Xiong, Richard Socher, Dragomir Radev
<span title="">2021</span> <i title="MIT Press - Journals"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/jtrizn7izbhdlfmye6eriw6ub4" style="color: black;">Transactions of the Association for Computational Linguistics</a> </i> &nbsp;
summarization model outputs along with expert and crowd-sourced human annotations; 2) we consistently benchmark 23 recent summarization models using the aforementioned automatic evaluation metrics; 3)  ...  and unified API for evaluating summarization models across a broad range of automatic metrics; and 5) we assemble and share the largest and most diverse, in terms of model types, collection of human judgments  ...  Model Re-evaluation We now turn to an analysis of model scores across human evaluations and automatic metrics.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1162/tacl_a_00373">doi:10.1162/tacl_a_00373</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/avokt36bprc6xmco2cii6672xe">fatcat:avokt36bprc6xmco2cii6672xe</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20210430132146/https://watermark.silverchair.com/tacl_a_00373.pdf?token=AQECAHi208BE49Ooan9kkhW_Ercy7Dm3ZL_9Cf3qfKAc485ysgAAAqkwggKlBgkqhkiG9w0BBwagggKWMIICkgIBADCCAosGCSqGSIb3DQEHATAeBglghkgBZQMEAS4wEQQMaPac9wXMVOQmfk2eAgEQgIICXOcuEFHDIYkAuWkZzLrm_B_EJq2kGmJUCUUlw5lQjiBztxjJU4uObT0yW_XLbKaF8wcs_FGNavBu3K5sQ7yqDdNpPSUmaB6O0fSGyEiBqk9olrHXiwGPlUozS5GUQpyro2k58qP1FgO7LFhhjJbxSBJW3HgnW2xuPWECM_VKULhLwzUpdmGYO0p4QLMkPIHEeySU2IKdNAcREIzCwpQ0TG78SCe--X8lHSG2bOKacsKFxT9MaJs7VERNu2hDy9r-PHvA4Ct4V7vB3I0HZNN4Ub2R8HAmeZ6s9eosqtlwhwxTox3-5N1OQpNLrmsDeiC1DaGrRZCavzUoHAGTGZh5ADMmtSNF9tYtyYdgPhd4XjMVttsm3lZ0vVLKLT5nqgJrWEW9obaohenVF82fjzGp23ZciMzsDNBHrHHdwJHPWsf2cOx0tuBZGhHTURMASaTJJid5WpVj8W1G5zaylKQogFFv2ZuOzfGRpIWwtChNv_rJv0m-C05fi4ytd2T6n6f57MOAB18xVsfAOeb_Ce8wZ5VuIhgnXXbImDZ0V4OLAV7f1LMhjitWtCfoOk37I9ft8qqOBDAPOwD6wRA19DR5cfeYTdbqIsPNWxdGSar647lR5wqi1C1DSn6nK0l1x_yxx2hiWNRpswnHEzAAqpvbZXWsj6ankvEkYzwYq5P8A-BnRQaDTPofDBuvX5yHNlBrTx_9uv_R3fqZ0cOJNjGEv0ubhTr5XpCEIrwY2bdzwrQjjhSa_foJmg9maZPjIYHhkJHEAKLIMTAfRJc6TE5GBhFKZ71nju1UyWX95U4" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/f9/86/f986352571ee960682bafaed08b1dbcd2d68e402.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1162/tacl_a_00373"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="unlock alternate icon" style="background-color: #fb971f;"></i> mitpressjournals.org </button> </a>
&laquo; Previous Showing results 1 &mdash; 15 out of 286,697 results