An Optimization Framework for Weighting Implicit Relevance Labels for Personalized Web Search

Yury Ustinovskiy, Gleb Gusev, Pavel Serdyukov
2015 Proceedings of the 24th International Conference on World Wide Web - WWW '15  
Implicit feedback from users of a web search engine is an essential source providing consistent personal relevance labels from the actual population of users. However, previous studies on personalized search employ this source in a rather straightforward manner. Basically, documents that were clicked on get maximal gain, and the rest of the documents are assigned the zero gain. As we demonstrate in our paper, a ranking algorithm trained using these gains directly as the ground truth relevance
more » ... bels leads to a suboptimal personalized ranking. In this paper we develop a framework for automatic reweighting of these labels. Our approach is based on more subtle aspects of user interaction with the result page. We propose an efficient methodology for deriving confidence levels for relevance labels that relies directly on the objective ranking measure. All our algorithms are evaluated on a large-scale query log provided by a major commercial search engine. The results of the experiments prove that the current state-of-the-art personalization approaches could be significantly improved by enriching relevance grades with weights extracted from post-impression user behavior.
doi:10.1145/2736277.2741105 dblp:conf/www/UstinovskiyGS15 fatcat:3lzngiwzyzhhheidg4wouoskji