Filters








6 Hits in 1.6 sec

UQV100

Peter Bailey, Alistair Moffat, Falk Scholer, Paul Thomas
2016 Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval - SIGIR '16  
We describe the UQV100 test collection, designed to incorporate variability from users.  ...  The backstories, query variations, normalized and spell-corrected queries, effort estimates, run outputs, and relevance judgments are made available collectively as the UQV100 test collection.  ...  Our purpose in this work is to address the challenge posed by this earlier work, and develop a test collection that explicitly includes query variability as a factor.  ... 
doi:10.1145/2911451.2914671 dblp:conf/sigir/BaileyMST16 fatcat:badjuucmcrd6hbbb47xszn4rju

Retrieval Consistency in the Presence of Query Variations

Peter Bailey, Alistair Moffat, Falk Scholer, Paul Thomas
2017 Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval - SIGIR '17  
In this paper we examine the retrieval consistency of a set of ve systems responding to syntactic query variations over one hundred topics, working with the UQV100 test collection, and using Rank-Biased  ...  RBC is compared with alternative data fusion algorithms. Our results indicate that consistency is positively correlated to a moderate degree with "deep" relevance measures.  ...  Ma Crane, Xiaolu Lu, David Maxwell, and Andrew Trotman assisted greatly, providing the system runs that were analyzed. e UQV100 judgments were generated using resources provided by Microso .  ... 
doi:10.1145/3077136.3080839 dblp:conf/sigir/BaileyMST17 fatcat:3vzdvhjyvnhjfpisvzvb66mrgi

Evaluating the Robustness of Retrieval Pipelines with Query Variation Generators [article]

Gustavo Penha, Arthur Câmara, Claudia Hauff
2022 arXiv   pre-print
tested with modified inputs.  ...  In order to obtain queries that are representative of users' querying variability, we first created a taxonomy based on the manual annotation of transformations occurring in a dataset (UQV100) of user-created  ...  Our work highlights the need of creating test collections that include query variations to better understand model effectiveness.  ... 
arXiv:2111.13057v3 fatcat:mgjubdswa5hdbhhiexbxcf6foq

Boosting Search Performance Using Query Variations [article]

Rodger Benham, Joel Mackenzie, Alistair Moffat, J. Shane Culpepper
2018 arXiv   pre-print
Experimental results using the ClueWeb12B collection and the UQV100 query variations demonstrate that centroid-based approaches allow improved retrieval effectiveness at little or no loss in query throughput  ...  in multi-stage retrieval systems with little efficiency overhead at query time.  ...  This work was supported by the Australian Research Council's Discovery Projects Scheme (DP170102231) and a grant from the Mozilla Foundation.  ... 
arXiv:1811.06147v1 fatcat:v2hx4h3olfh63bqyaocs5v7ssa

RMIT at the 2018 TREC CORE Track

Rodger Benham, Luke Gallagher, Joel Mackenzie, Binsheng Liu, Xiaolu Lu, Falk Scholer, J. Shane Culpepper, Alistair Moffat
2018 Text Retrieval Conference  
Our thesis is that over-reliance on a single query can lead to suboptimal performance, and that by creating multiple query representations for an information need and combining the relevance signals through  ...  Ad-hoc retrieval is an important problem with many practical applications.  ...  [15] showed that test collections formed without user query variability do not generalize well outside of the supplied title query.  ... 
dblp:conf/trec/BenhamGML0SCM18 fatcat:wo453fdgnnbnxnwwv3hba6f7pm

A Study of a Gain Based Approach for Query Aspects in Recall Oriented Tasks

Giorgio Maria Di Nunzio, Guglielmo Faggioli
2021 Applied Sciences  
We evaluate and propose alternative formulations of this metric using standard test collections of the CLEF 2018 eHealth Evaluation Lab CHS.  ...  In this context, the task of collecting all the relevant information, a recall oriented task, in order to take the right decision within a reasonable time frame has become an important issue.  ...  In particular, we plan to include new collections, topics and formulations. We plan to investigate the news domain through multiple formulations available in the UQV100 collection [1] .  ... 
doi:10.3390/app11199075 fatcat:rwdaosmmx5cepmn3vkbv42ickq