316 Hits in 3.5 sec

Overview of the TREC 2018 Real-Time Summarization Track

Royal Sequiera, Luchen Tan, Jimmy Lin
2018 Text Retrieval Conference  
INTRODUCTION e TREC 2018 Real-Time Summarization (RTS) Track is the third iteration of a community e ort to explore techniques, algorithms, and systems that automatically monitor streams of social media  ...  CONCLUSIONS is year represents the nal year of the Real-Time Summarization Track, the culmination of a journey that began with the Microblog Track at TREC 2011.  ... 
dblp:conf/trec/SequieraTL18 fatcat:xw3yjcyuwbcornffc342cpu36m

Report on the Second SIGIR Workshop on Neural Information Retrieval (Neu-IR'17)

Nick Craswell, W. Bruce Croft, Maarten de Rijke, Jiafeng Guo, Bhaskar Mitra
2018 SIGIR Forum  
Organizers of four of the TREC 2017 tracks were invited to present at the workshop on how these IR tasks may be suitable for evaluating recent data-hungry neural approaches.  ...  A total of 19 papers-which included five special track paperswere presented in the form of oral or poster presentations.  ...  TREC track Speaker Website Complex Answers Retrieval Laura Dietz Real-Time Summarization Jimmy Lin Dynamic Domain Grace Hui Yang  ... 
doi:10.1145/3190580.3190603 fatcat:23pwl6jtnbdh3ifxexlwqozdhu

TREC Incident Streams: Finding Actionable Information on Social Media

Richard McCreadie, Cody Buntain, Ian Soboroff
2019 International Conference on Information Systems for Crisis Response and Management  
The 2018 edition of the track provides a standardized evaluation methodology, an ontology of emergency-relevant social media information types, proposes a scale for information criticality, and releases  ...  The Text Retrieval Conference (TREC) Incident Streams track is a new initiative that aims to mature social media-based emergency response technology.  ...  Providing the primary overview for the 2018 edition of the track, detailing track design and motivating why core design decisions were made. 2.  ... 
dblp:conf/iscram/McCreadieBS19 fatcat:lvkg2v4flzak3p5xkoqwh7jaoa

Optimization framework model for retrospective tweet summarization

Abdelhamid Chellal, Mohand Boughanem
2018 Proceedings of the 33rd Annual ACM Symposium on Applied Computing - SAC '18  
However, timely following the development of long-running events is too difficult due to the velocity and the volume of the published information.  ...  Our experiments on TREC RTF 2015 and TREC RTS 2016 datasets have shown the effectiveness of our approach.  ...  To evaluate the proposed approach, we carried out several experiments on TREC Microblog Real-Time Filtering 2015 (MB-RTF) dataset [7] and TREC Real-Time summarization (RTS) 2016 track [8] .  ... 
doi:10.1145/3167132.3167210 dblp:conf/sac/ChellalB18 fatcat:frjg6cxs6regpkkmk57s3ofweu

Explicit Diversification of Event Aspects for Temporal Summarization

Richard McCreadie, Rodrygo L. T. Santos, Craig Macdonald, Iadh Ounis
2018 ACM Transactions on Information Systems  
Moreover, much of the information reported is redundant and/or out-of-date [18] . Hence, there is a need for automatic approaches that summarize that content for the user in real-time.  ...  Temporal summarization (TS) approaches are used to automatically produce concise overviews of such events, by extracting text snippets from related articles over time.  ...  ACKNOWLEDGMENTS This work is supported by the EC co-funded SUPER (FP7-606853) project.  ... 
doi:10.1145/3158671 fatcat:fw4hyf2j5fccpldlgpfzq4bnra

Overview of the 2019 Open-Source IR Replicability Challenge (OSIRRC 2019)

Ryan Clancy, Nicola Ferro, Claudia Hauff, Jimmy Lin, Tetsuya Sakai, Ze Zhong Wu
2019 Annual International ACM SIGIR Conference on Research and Development in Information Retrieval  
In total, 13 teams from around the world submitted 17 images, most of which were designed to produce retrieval runs for the TREC 2004 Robust Track test collection.  ...  We articulate the goals of this workshop and describe the "jig" that encodes the Docker specification.  ...  ACKNOWLEDGEMENTS We would like to thank all the participant who contributed Docker images to the workshop. This exercise would not have been possible without their efforts.  ... 
dblp:conf/sigir/Clancy0HLSW19a fatcat:qi5ug6dgejakvoipfa5hsgqo3m

Overview of the TREC 2017 Real-Time Summarization Track

Jimmy Lin, Salman Mohammed, Royal Sequiera, Luchen Tan, Nimesh Ghelani, Mustafa Abualsaud, Richard McCreadie, Dmitrijs Milajevs, Ellen M. Voorhees
2017 Text Retrieval Conference  
ACKNOWLEDGMENTS is work was supported in part by the Natural Sciences and Engineering Research Council (NSERC) of Canada.  ...  We'd like to thank all the mobile assessors who participated in our user study and the NIST assessors.  ...  EVALUATION DESIGN 2.1 General Setup e overall design of the TREC 2017 Real-Time Summarization Track followed the iteration of the track in TREC 2016 [8] , which was itself adapted from the real-time ltering  ... 
dblp:conf/trec/LinMSTGAMMV17 fatcat:iagtggaonvai5hc7v3tgrccqe4

Component-Based Evaluation for Question Answering [chapter]

Teruko Mitamura, Eric Nyberg
2020 Evaluating Information Retrieval and Access Tasks  
We summarize the history of component evaluation for QA and describe more recent work at Carnegie Mellon (on TREC Genomics, BioASQ, and LiveQA datasets) which has descended directly from our experiences  ...  This chapter describes the component-based evaluation of automatic question answering (QA) systems, which was pioneered in the NTCIR-7 ACLIA challenge and has became a fundamental part of QA system development  ...  Acknowledgements We would like to thank to the editors and to the past organizers and participants of the NTCIR ACLIA QA tasks.  ... 
doi:10.1007/978-981-15-5554-1_8 fatcat:3pklke6ulvh7xjlxwxxzctgfue

Evaluation in Contextual Information Retrieval

Lynda Tamine, Mariam Daoud
2018 ACM Computing Surveys  
While evaluation is central and has a long history in information retrieval, it faces the big challenge of designing an appropriate methodology that embeds the context into evaluation settings.  ...  Context such as the user's search history, demographics, devices, and surroundings, has become prevalent in various domains of information seeking and retrieval such as mobile search, task-based search  ...  The main task in TREC 2011 through TREC 2014 consists of a real-time ad hoc search task where context data includes the query time that represents the timestamp of the query in a human and machine readable  ... 
doi:10.1145/3204940 fatcat:qzvhpa6rlvhh5hrsbblhiulwvy

Everything You Always Wanted to Know About TREC RTS* (*But Were Afraid to Ask) [article]

Gilles Hubert, Jose G. Moreno, Karen Pinel-Sauvagnat, Yoann Pitarch
2017 arXiv   pre-print
The TREC Real-Time Summarization (RTS) track provides a framework for evaluating systems monitoring the Twitter stream and pushing tweets to users according to given profiles.  ...  In this work, we perform a thorough analysis of each component of the framework used in 2016 and 2017 and found some limitations for the Scenario A of this track.  ...  The TREC campaign took an interest in the evaluation of such systems through various tracks and notably the ongoing Real-Time Summarization (RTS) track.  ... 
arXiv:1712.04671v1 fatcat:wyoh6spsczfyth3m7cpps7jn4i

Community challenges in biomedical text mining over 10 years: success, failure and the future

Chung-Chi Huang, Zhiyong Lu
2015 Briefings in Bioinformatics  
Finally, we summarize the impact and contributions by taking into account different BioNLP challenges as a whole, followed by a discussion of their limitations and difficulties.  ...  One effective way to improve the state of the art is through competitions.  ...  Funding This work was supported by the National Institutes of Health intramural research program, National Library of Medicine.  ... 
doi:10.1093/bib/bbv024 pmid:25935162 pmcid:PMC4719069 fatcat:z7wlyvovnnbrrkufmqaxycqdyi

Some thoughts from IRIT about the scenario A of the TREC RTS 2016 and 2017 tracks

Gilles Hubert, Jose G. Moreno, Karen Pinel-Sauvagnat, Yoann Pitarch
2017 Text Retrieval Conference  
The TREC Real-Time Summarization (RTS) track provides a framework for evaluating systems monitoring the Twitter stream and pushing tweets to users according to given profiles.  ...  This is an extended version of our paper "Everything You Always Wanted to Know About TREC RTS* (*But Were Afraid to Ask)" [3].  ...  The TREC campaign took an interest in the evaluation of such systems through various tracks and notably the ongoing Real-Time Summarization (RTS) track.  ... 
dblp:conf/trec/HubertMPP17 fatcat:vnwuu3p4czh33ne3y2tcrguepm

JULIE Lab & Med Uni Graz @ TREC 2019 Precision Medicine Track

Erik Faessler, Udo Hahn, Michel Oleynik
2019 Text Retrieval Conference  
The 2019 Precision Medicine Track at TREC (TREC-PM) aimed at identifying relevant documents from two collections, namely PubMed (biomedical abstracts) and (clinical trials), given 40  ...  Our approach builds upon carefully designed weighted queries based on our experience from last year's participation and explores the usefulness of Learning to Rank (LETOR), trained on either the previous  ...  The contributions of the first and third author were funded by the German Bundesministerium für Bildung und Forschung (BMBF) under grant no. 01ZZ1803G.  ... 
dblp:conf/trec/FaesslerHO19 fatcat:xshzxnt5t5gfrb2kf3octtne2i

Exploiting Transfer Learning and Hand-Crafted Features in a Unified Neural Model for Identifying Actionable Informative Tweets

Abu Nowshed Chy, Umme Aymun Siddiqua, Masaki Aono
2021 Journal of Information Processing  
Experiments on the 2018 TREC incident streams (TREC-IS) dataset show that our proposed neural model learns the contextual information effectively and achieves the overall best result compared to the state-of-the-art  ...  On top of nested LSTMs, we perform the convolution using multiple kernels (CMK) to obtain the higher-level representation of tweets.  ...  , and the top-performing system of the 2018 TREC-IS track cbnuS2 [5] in Table 9 .  ... 
doi:10.2197/ipsjjip.29.16 fatcat:clx7e7s7vjdspgjdilwrzzthom

Applying BERT to Document Retrieval with Birch

Zeynep Akkalyoncu Yilmaz, Shengjin Wang, Wei Yang, Haotian Zhang, Jimmy Lin
2019 Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP): System Demonstrations  
Birch implements simple ranking models that achieve stateof-the-art effectiveness on standard TREC newswire and social media test collections.  ...  This demonstration focuses on technical challenges in the integration of NLP and IR capabilities, along with the design rationale behind our approach to tightly-coupled integration between Python (to support  ...  Acknowledgments This research was supported by the Natural Sciences and Engineering Research Council (NSERC) of Canada, and enabled by computational resources provided by Compute Ontario and Compute Canada  ... 
doi:10.18653/v1/d19-3004 dblp:conf/emnlp/YilmazWYZL19 fatcat:h65c4vioy5hzdm2p4njed4nqaq
« Previous Showing results 1 — 15 out of 316 results