Filters








291 Hits in 3.6 sec

TREC 2017 Dynamic Domain Track Overview

Grace Hui Yang, Zhiwen Tang, Ian Soboroff
2017 Text Retrieval Conference  
Discussion The dynamic domain track has been running in the past three years at TREC and this is the final year.  ...  Sample topic of TREC 2017 DD track Figure 2 . 2 Figure 2. Sample Relevance Judgement of TREC 2017 DD track Figure 3 .Figure 4 . 34 Figure 3. DD track Task illustration are provided.  ... 
dblp:conf/trec/YangTS17 fatcat:3oftstztnzcozprly2zt2frpha

CLEF 2017 Dynamic Search Evaluation Lab Overview [chapter]

Evangelos Kanoulas, Leif Azzopardi
2017 Lecture Notes in Computer Science  
In this paper we provide an overview of the first edition of the CLEF Dynamic Search Lab.  ...  Herein, we describe the overall objectives of the CLEF 2017 Dynamic Search Lab, the resources created for the pilot task, the evaluation methodology adopted, and some preliminary evaluation results of  ...  Lab Overview The CLEF Dynamic Tasks Lab attempts to focus attention towards building a bridge between batch TREC-style evaluation methodology and the Interactive Information Retrieval evaluation methodology  ... 
doi:10.1007/978-3-319-65813-1_31 fatcat:bczfn7xtvbgivgyppwm7ka7xxy

UMD_CLIP: Using Relevance Feedback to Find Diverse Documents for TREC Dynamic Domain 2017

Kristine Rogers, Douglas W. Oard
2017 Text Retrieval Conference  
The University of Maryland's participation in TREC's 2017 Dynamic Domain track focused on two types of experiments: adding new terms from passages judged as being relevant, and exclusion of terms from  ...  Introduction The participation of the Computational Linguistics and Information Processing Lab at the University of Maryland (UMD_CLIP) in TREC 2017 Dynamic Domain track (TREC-DD) focused on characterizing  ...  Negative Feedback We tried some negative feedback approaches that were motivated generally by an approach described in the TREC 2015 Dynamic Domain track overview (Yang & Soboroff, 2015) that was attributed  ... 
dblp:conf/trec/RogersO17 fatcat:txsx6avpkbdwtdv6bhfkkrtzhi

Overview of the CLEF Dynamic Search Evaluation Lab 2018 [chapter]

Evangelos Kanoulas, Leif Azzopardi, Grace Hui Yang
2018 Lecture Notes in Computer Science  
In this paper we provide an overview of the CLEF 2018 Dynamic Search Lab. The lab ran for the first time in 2017 as a workshop.  ...  The lab strives to answer one key question: how can we evaluate, and consequently build, dynamic search algorithms?  ...  Potential participants are provided with a RESTful API to query the index 3 . dd17- The topics used are the topics developed by NIST assessors for the TREC 2017 Dynamic Domain Track [19] .  ... 
doi:10.1007/978-3-319-98932-7_31 fatcat:hbwbpbngvfbexoif34prsout4i

Report on the Second SIGIR Workshop on Neural Information Retrieval (Neu-IR'17)

Nick Craswell, W. Bruce Croft, Maarten de Rijke, Jiafeng Guo, Bhaskar Mitra
2018 SIGIR Forum  
Organizers of four of the TREC 2017 tracks were invited to present at the workshop on how these IR tasks may be suitable for evaluating recent data-hungry neural approaches.  ...  The second SIGIR workshop on neural information retrieval (Neu-IR'17) took place on August 11, 2017, in Tokyo, Japan.  ...  TREC track Speaker Website Complex Answers Retrieval Laura Dietz http://trec-car.cs.unh.edu/ Real-Time Summarization Jimmy Lin http://trecrts.github.io/ Dynamic Domain Grace Hui Yang http://trec-dd.org  ... 
doi:10.1145/3190580.3190603 fatcat:23pwl6jtnbdh3ifxexlwqozdhu

Corpus-Level End-to-End Exploration for Interactive Systems [article]

Zhiwen Tang, Grace Hui Yang
2019 arXiv   pre-print
Experiments on the Text REtrieval Conference (TREC) Dynamic Domain (DD) Track show that CE3 outperforms the state-of-the-art DS systems.  ...  One example is Dynamic Search (DS), which models the process that a human works with a search engine agent to accomplish a complex and goal-oriented task.  ...  The Dynamic Domain (DD) Tracks held at the Text REtrieval Conference (TREC) from 2015 to 2017 (Yang, Tang, and Soboroff 2017) are campaigns that evaluate DS systems.  ... 
arXiv:1912.00753v1 fatcat:j26a4vb6ozgltagx3xdafpytei

Corpus-Level End-to-End Exploration for Interactive Systems

Zhiwen Tang, Grace Hui Yang
2020 PROCEEDINGS OF THE THIRTIETH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE AND THE TWENTY-EIGHTH INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE  
Experiments on the Text REtrieval Conference (TREC) Dynamic Domain (DD) Track show that CE3 outperforms the state-of-the-art DS systems.  ...  One example is Dynamic Search (DS), which models the process that a human works with a search engine agent to accomplish a complex and goal-oriented task.  ...  The Dynamic Domain (DD) Tracks held at the Text REtrieval Conference (TREC) from 2015 to 2017 (Yang, Tang, and Soboroff 2017) are campaigns that evaluate DS systems.  ... 
doi:10.1609/aaai.v34i03.5635 fatcat:ac6f5qfdqnasxhuyihopnldsxe

GSMaP RIKEN Nowcast: Global Precipitation Nowcasting with Data Assimilation

Shigenori OTSUKA, Shunji KOTSUKI, Marimo OHHIGASHI, Takemasa MIYOSHI
2019 Journal of the Meteorological Society of Japan  
C2b).691From 10 February 2017, d 0 is set to 40 pixels with R 0 = 10 −5 mm h −1 .692 C.3 Smoothing rain field in TREC 693 Since 29 November 2017, rain fields are spatially smoothed during the 694 computation  ...  Overview of the system updates 556 • 567 •• 567 V1 suffered from lack of information over weak rain areas.  ... 
doi:10.2151/jmsj.2019-061 fatcat:jmfu67kil5djzovmj73fhkcja4

Searching for Scientific Evidence in a Pandemic: An Overview of TREC-COVID [article]

Kirk Roberts, Tasmeer Alam, Steven Bedrick, Dina Demner-Fushman, Kyle Lo, Ian Soboroff, Ellen Voorhees, Lucy Lu Wang, William R Hersh
2021 arXiv   pre-print
This paper provides a comprehensive overview of the structure and results of TREC-COVID.  ...  We present an overview of the TREC-COVID Challenge, an information retrieval (IR) shared task to evaluate search on scientific literature related to COVID-19.  ...  The TREC Precision Medicine track (2017-2020) [19] [20] [21] [22] refined that focus to oncologists interested in treating cancer patients with actionable gene mutations.  ... 
arXiv:2104.09632v1 fatcat:fy24bnbejvgx5oiwcty6ty2vpi

Transformer-based Multi-task Learning for Disaster Tweet Categorisation [article]

Congcong Wang, Paul Nulty, David Lillis
2021 arXiv   pre-print
We evaluate the effectiveness of our approach with a variety of metrics by submitting runs to the TREC Incident Streams (IS) track: a research initiative specifically designed for disaster tweet classification  ...  We make the code publicly available so that our work can be reproduced and used as a baseline for the community for future work in this domain.  ...  RQ3: TREC-IS track participation We have reported that our MTL-based runs have an advantage over both the BERT-based and LR-based STL baselines.  ... 
arXiv:2110.08010v1 fatcat:t6eu2ujecnfpvlbjwue3aelnnm

UTD HLTRI at TREC 2017: Complex Answer Retrieval Track

Ramón Maldonado, Stuart J. Taylor, Sanda M. Harabagiu
2017 Text Retrieval Conference  
This paper presents our Complex Answer PAragraph Retrieval (CAPAR) system designed for our participation in the TREC Complex Answer Retrieval (CAR) track.  ...  Overview of Runs We submitted three runs to TREC CAR 2017.  ...  ."), in 2017 the Text REtrieval Conference Complex Answer Retrieval (TREC CAR) track was designed to evaluate systems performing open-domain, complex answer retrieval.  ... 
dblp:conf/trec/MaldonadoTH17 fatcat:4mjvepban5bgllnmxuxcyayi5y

BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models [article]

Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, Iryna Gurevych
2021 arXiv   pre-print
We leverage a careful selection of 18 publicly available datasets from diverse text retrieval tasks and domains and evaluate 10 state-of-the-art retrieval systems including lexical, sparse, dense, late-interaction  ...  D.4 News Retrieval TREC-NEWS [58] 2019 track involves background linking: Given a news headline, we retrieve relevant news articles that provide important context or background information.  ...  . ‡ indicates in-domain retrieval performance. shows the capped Recall@100 score (Appendix G). Figure 1 : 1 An overview of the diverse tasks and datasets in BEIR benchmark.  ... 
arXiv:2104.08663v4 fatcat:fow5uqghbzggjclobtv7bpjtaa

Summarizing Microblogs for Emergency Relief and Preparedness

Kanav Mehra, Vibhash Chandra
2017 European Conference on Information Retrieval  
The result of our participation in the SMERP 2017 Data Challenge Track shows that it is an effective approach in summarizing tweets in a disaster scenario and can be replicated across diverse domains.  ...  The SMERP 2017 Data Challenge Track is motivated by this scenario, and aspires to endorse the development of dynamic and adaptive summarization system to provide meaningful insights from microblogs posted  ...  Methodology Enhancement in Level-2: In the SMERP 2017 Data Challenge Track, organizers have released the data in two levels as illustrated in Section 2.  ... 
dblp:conf/ecir/MehraC17 fatcat:uqrs52b7lzgdth2mvwjthvc6lq

Navigating Imprecision in Relevance Assessments on the Road to Total Recall

Gordon V. Cormack, Maura R. Grossman
2017 Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval - SIGIR '17  
The results using the TREC 4 data show that TAR achieves higher recall and higher precision than the assessments by either of two independent NIST assessors, and blind adjudication of the email dataset  ...  This hypothesis is tested by simulating TAR using two datasets: the TREC 4 AdHoc collection, and a dataset consisting of 401,960 email messages that were manually reviewed and classified by a single individual  ...  It is important to bear in mind that results measuring system recall, rather than end-to-end recall, cannot be compared to the results reported here, to those reported in the TREC 2009 Legal Track Overview  ... 
doi:10.1145/3077136.3080812 dblp:conf/sigir/CormackG17 fatcat:bnvqc7zwkjaztcqo5taev3bsvm

Challenges in Patent Information Retrieval [chapter]

Makoto Iwayama, Atsushi Fujii, Hidetsugu Nanba
2020 Evaluating Information Retrieval and Access Tasks  
of the TREC 2009 chemical IR track.  ...  TREC-CHEM also had tasks for prior art search and technology survey. The TREC-CHEM tasks were challenging and focused on the chemical domain, which has many formulae and images.  ... 
doi:10.1007/978-981-15-5554-1_4 fatcat:pqchrs6jcfd6lcomgzj5gzegb4
« Previous Showing results 1 — 15 out of 291 results