Challenging conventional assumptions of automated information retrieval with real users: Boolean searching and batch retrieval evaluations

William Hersh, Andrew Turpin, Susan Price, Dale Kraemer, Daniel Olson, Benjamin Chan, Lynetta Sacherek
2001 Information Processing & Management  
Two common assumptions held by information retrieval researchers are that searching using Boolean operators is inferior to natural language searching and that results from batch-style retrieval evaluations are generalizable to the real-world searching. We challenged these assumptions in the Text Retrieval Conference (TREC) interactive track, with real users following a consensus protocol to search for an instance recall task. Our results showed that Boolean and natural language searching
more » ... d comparable results and that the results from batch evaluations were not comparable to those obtained in experiments with real users. Ó
doi:10.1016/s0306-4573(00)00054-6 fatcat:s4fad5xgfzerzasxkqqzrkqobm