Comparing the performance of database selection algorithms

James C. French, Allison L. Powell, Jamie Callan, Charles L. Viles, Travis Emmitt, Kevin J. Prey, Yun Mou
1999 Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval - SIGIR '99  
We compare the performance of two database selection algorithms reported in the literature. Their performance is compared using a common testbed designed specifically for database selection techniques. The testbed is a decomposition of the TREC TIPSTER data into 236 subcollections. The databases from our testbed were ranked using both the gGlOSS and CORI techniques and compared to a baseline derived from TREC relevance judgements. We examined the degree to which CORI and gGlOSS approximate this
more » ... baseline. Our results con rm our earlier observation that the gGlOSS Ideall ranks do not estimate relevancebased ranks well. We also nd that CORI is a uniformly better estimator of relevance-based ranks than gGlOSS for the test environment used in this study. Part of the advantage of the CORI algorithm can be explained by a strong correlation between gGlOSS and a size-based baseline SBR. We also nd that CORI produces consistently accurate rankings on testbeds ranging from 100 921 sites. However for a given level of recall, search e ort appears to scale linearly with the number of databases.
doi:10.1145/312624.312684 dblp:conf/sigir/FrenchPCVEPM99 fatcat:zuuo7k63kbdv5l5tjgdpxk4wri