Experiences evaluating personal metasearch

Paul Thomas, David Hawking
2008 Proceedings of the second international symposium on Information interaction in context - IIiX '08  
Many current evaluation techniques for information retrieval, such as test collections and simulations, are difficult to apply in situations where queries and preferred results are context-dependent. This is particularly true in personal metasearch applications, which provide a person with unified search access to all their usual online sources. A recently-proposed technique, based on presenting two or more search results sets in a single comparison interface, offers an alternative. We have
more » ... dded this technique in a working personal metasearch tool which we have distributed to volunteers. Initial experiments with server selection suggest that the technique is accepted by users, can operate over diverse and unarticulated contexts, and that the data it provides can provide a useful comparison to that from test collections. Further experimentation with the technique is continuing.
doi:10.1145/1414694.1414724 dblp:conf/iiix/ThomasH08 fatcat:kz7n5dd3qvblrmytqxvdfkwfu4