Filters








322,314 Hits in 5.7 sec

Evaluation challenges in large-scale document summarization

Dragomir R. Radev, Simone Teufel, Horacio Saggion, Wai Lam, John Blitzer, Hong Qi, Arda Çelebi, Danyu Liu, Elliott Drabek
2003 Proceedings of the 41st Annual Meeting on Association for Computational Linguistics - ACL '03  
We present a large-scale meta evaluation of eight evaluation measures for both single-document and multi-document summarizers.  ...  To this end we built a corpus consisting of (a) 100 Million automatic summaries using six summarizers and baselines at ten summary lengths in both English and Chinese, (b) more than 10,000 manual abstracts  ...  Although they can be a very effective way of measuring summary quality, task-based evaluations are prohibitively expensive at large scales.  ... 
doi:10.3115/1075096.1075144 dblp:conf/acl/RadevTSLBQCLD03 fatcat:3hssojx25jhwlgr52u7b36aiiy

Multi-XScience: A Large-scale Dataset for Extreme Multi-document Summarization of Scientific Articles [article]

Yao Lu, Yue Dong, Laurent Charlin
2020 arXiv   pre-print
Multi-document summarization is a challenging task for which there exists little large-scale datasets.  ...  We propose Multi-XScience, a large-scale multi-document summarization dataset created from scientific articles.  ...  Introduction Single document summarization is the focus of most current summarization research thanks to the availability of large-scale single-document summarization datasets spanning multiple fields,  ... 
arXiv:2010.14235v1 fatcat:baas5st355fy7o4nqedrhwrb6a

Big Data Summarization : Framework, Challenges and Possible Solutions

Shilpa G. Kolte, Jagdish W. Bakal
2016 Advanced Computational Intelligence An International Journal (ACII)  
The objective of this paper is to discuss the big data summarization framework, challenges and possible solutions as well as methods of evaluation for big data summarization.  ...  In this paper, we first briefly review the concept of big data, including its definition, features, and value. We then present background technology for big data summarization brings to us.  ...  BIG DATA SUMMARIZATION FRAMEWORK, CHALLENGE, AND POSSIBLE SOLUTION Big data not only refers to datasets that are large in size, but also covers datasets that are complex in structures, high dimensional  ... 
doi:10.5121/acii.2016.3401 fatcat:jshfr2clf5cgxjwz7zbmnfz3n4

Evaluation of Information Access Technologies at the NTCIR Workshop [chapter]

Noriko Kando
2004 Lecture Notes in Computer Science  
The importance of large-scale evaluation infrastructure in IA research has been widely recognized.  ...  question answering, text mining, etc., by providing infrastructure of large-scale evaluation.  ...  ., based on the information obtained from the retrieved documents. We have looked at IA technologies to help users utilize the information in large-scale document collections.  ... 
doi:10.1007/978-3-540-30222-3_4 fatcat:kox5oyahanh2hlhybiwfct6rce

NTCIR-4: Outline of Invited Talk at CLEF 2004 Workshop

Noriko Kando
2004 Conference and Labs of the Evaluation Forum  
(IR), crosslingual information retrieval (CLIR), automatic text summarization, question answering, text mining and so on by providing large-scale test collections and a forum for researchers.  ...  This talk will present the fourth NTCIR Workshop, which is the latest in a series of evaluation workshops designed to enhance research in information access (IA) technologies including information retrieval  ...  Relevance judgments were done at both document-and passage-levels. TSC included the automatic evaluation of summaries and the building of a re-usable test collection for summarization.  ... 
dblp:conf/clef/Kando04 fatcat:vll6nugxszf4vphz6uzm6ir4tu

Recent developments in text summarization

Inderjeet Mani
2001 Proceedings of the tenth international conference on Information and knowledge management - CIKM'01  
In this paper, I will discuss the significance of some recent developments in summarization technology.  ...  With the explosion in the quantity of on-line text and multimedia information in recent years, demand for text summarization technology is growing.  ...  Last, but not least, there has been increasing activity in summarization evaluation, with several large-scale evaluations being carried out.  ... 
doi:10.1145/502585.502677 dblp:conf/cikm/Mani01 fatcat:iwewickpdjgdrkz5nkolp7ef4e

Recent developments in text summarization

Inderjeet Mani
2001 Proceedings of the tenth international conference on Information and knowledge management - CIKM'01  
In this paper, I will discuss the significance of some recent developments in summarization technology.  ...  With the explosion in the quantity of on-line text and multimedia information in recent years, demand for text summarization technology is growing.  ...  Last, but not least, there has been increasing activity in summarization evaluation, with several large-scale evaluations being carried out.  ... 
doi:10.1145/502676.502677 fatcat:ddysknmd2rbaziiaawkemqsn3a

Development of a Konkani Language Dataset for Automatic Text Summarization and its Challenges

Jovi D'Silva, Uzzal Sharma
2019 Zenodo  
Automatic Text summarization attempts to automate the summarization task, which would otherwise, be done by humans. Research has progressed a lot in the said domain in languages such as English.  ...  Text summarization has gained tremendous popularity in the research field over the last few years.  ...  We would like to thank the human summarizers, Ms. Shubha Barad, Ms. Teffany Gama and Ms. Disha Mashelkar, and lastly, Mr. Rohan Kerkar for his valuable time and expertise in formatting the dataset.  ... 
doi:10.5281/zenodo.5531954 fatcat:4jygmgcqmrhorlpethkaiiuvxq

Editorial for the 3rd Joint Workshop on Bibliometric-enhanced Information Retrieval and Natural Language Processing for Digital Libraries (BIRNDL) at SIGIR 2018

Philipp Mayr, Muthu Kumar Chandrasekaran, Kokil Jaidka
2018 Annual International ACM SIGIR Conference on Research and Development in Information Retrieval  
All PC members are documented on the BIRNDL website 8 .  ...  Acknowledgments We thank Microsoft Research Asia for their generous support in funding the development, dissemination and organization of the CL-SciSumm dataset and the Shared Task 7 .  ...  This is the first medium-scale shared task on scientific document summarization in the computational linguistics (CL) domain.  ... 
dblp:conf/sigir/MayrCJ18 fatcat:75zb57nykres7m43k3c4q6kvkq

Page 298 of Computational Linguistics Vol. 31, Issue 3 [page]

2005 Computational Linguistics  
In particular, generation for sentence fusion must be able to operate in a domain- independent fashion, scalable to handle a large variety of input documents with various degrees of overlap.  ...  If language generation can be scaled to take fully formed text as input without semantic interpretation, selecting content and producing well-formed English sentences as output, then generation has a large  ... 

Managing the Knowledge Creation Process of Large-Scale Evaluation Campaigns [chapter]

Marco Dussin, Nicola Ferro
2009 Lecture Notes in Computer Science  
This paper discusses the evolution of large-scale evaluation campaigns and the corresponding evaluation infrastructures needed to carry them out.  ...  We present the next challenges for these initiatives and show how digital library systems can play a relevant role in supporting the research conducted in these fora by acting as virtual research environments  ...  The authors would like to thank Maristella Agosti and Giorgio Maria Di Nunzio for the useful discussions on the topics addressed in this chapter.  ... 
doi:10.1007/978-3-642-04346-8_8 fatcat:unitl3ft7rdp3aykpujrpw5hyu

AQuaMuSe: Automatically Generating Datasets for Query-Based Multi-Document Summarization [article]

Sayali Kulkarni, Sheide Chammas, Wan Zhu, Fei Sha, Eugene Ie
2020 arXiv   pre-print
summarization datasets are inadequate in form and scale.  ...  Query-based multi-document summarization (qMDS) addresses this pervasive need, but the research is severely limited due to lack of training and evaluation datasets as existing single-document and multi-document  ...  But only a few large-scale high-quality human-curated summarization datasets available for training and evaluation.  ... 
arXiv:2010.12694v1 fatcat:v6cgex7ttfhxzkiz2hl2d6ijxm

Challenges of developing a digital scribe to reduce clinical documentation burden

Juan C. Quiroz, Liliana Laranjo, Ahmet Baki Kocaballi, Shlomo Berkovsky, Dana Rezazadegan, Enrico Coiera
2019 npj Digital Medicine  
This paper identifies and discusses major challenges associated with developing automated speech-based documentation in clinical settings: recording high-quality audio, converting audio to transcripts  ...  Clinicians spend a large amount of time on clinical documentation of patient encounters, often impacting quality of care and clinician satisfaction, and causing physician burnout.  ...  CHALLENGE 3: INFORMATION EXTRACTION IN CLINICAL CONVERSATIONS Large-scale semantic taxonomies, such as the Unified Medical Language System (UMLS), allow for the identification of medical terminology in  ... 
doi:10.1038/s41746-019-0190-1 pmid:31799422 pmcid:PMC6874666 fatcat:y7owfblwlvc2dowldwf4c6pqbe

Beyond Opinion Mining: Summarizing Opinions of Customer Reviews [article]

Reinald Kim Amplayo, Arthur Bražinskas, Yoshi Suhara, Xiaolan Wang, Bing Liu
2022 arXiv   pre-print
In this tutorial, we present various aspects of opinion summarization that are useful for researchers and practitioners. First, we will introduce the task and major challenges.  ...  This three-hour tutorial will provide a comprehensive overview over major advances in opinion summarization.  ...  He has published extensively in top conferences and journals. He also authored four books about lifelong learning, sentiment analysis and Web mining.  ... 
arXiv:2206.01543v1 fatcat:axbzpqumobfzxdwvfnx2fwrvvu

More Than Reading Comprehension: A Survey on Datasets and Metrics of Textual Question Answering [article]

Yang Bai, Daisy Zhe Wang
2022 arXiv   pre-print
In this paper, we survey 47 recent textual QA benchmark datasets and propose a new taxonomy from an application point of view. In addition, We summarize 8 evaluation metrics of textual QA tasks.  ...  In recent years, many novel datasets and evaluation metrics based on classical MRC tasks have been proposed for broader textual QA tasks.  ...  Like CBT, accuracy is used as the evaluation metric. [37] is a Chinese large-scale cloze-style MRC dataset proposed by iFLYTEK Research in China.  ... 
arXiv:2109.12264v2 fatcat:sesgmfxagzdjji37cj3oin7yma
« Previous Showing results 1 — 15 out of 322,314 results