A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2015; you can also visit the original URL.
The file type is application/pdf
.
User Variability and IR System Evaluation
2015
Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval - SIGIR '15
Test collection design eliminates sources of user variability to make statistical comparisons among information retrieval (IR) systems more affordable. Does this choice unnecessarily limit generalizability of the outcomes to real usage scenarios? We explore two aspects of user variability with regard to evaluating the relative performance of IR systems, assessing effectiveness in the context of a subset of topics from three TREC collections, with the embodied information needs categorized
doi:10.1145/2766462.2767728
dblp:conf/sigir/BaileyMST15
fatcat:ctaphtd565akbofhoxxkl36cwq