Optimizing parameters of the expected reciprocal rank

Yury Logachev, Pavel Serdyukov
2012 Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval - SIGIR '12  
Most popular IR metrics are parameterized. Usually parameters of these metrics are chosen on the basis of general considerations and not adjusted by experiments with real users. Particularly, the parameters of the Expected Reciprocal Rank measure are the normalized parameters of the DCG metric, and the latter are chosen in an ad-hoc manner. We suggest an approach for adjusting parameters of the ERR metric that allows to reach maximum agreement with the real users behavior. More exactly, we
more » ... re exactly, we optimized the parameters by maximizing Pearson weighted correlation between ERR and several online click metrics. For each click metric we managed to find the parameters of ERR that result into its higher correlation with the given online click metric.
doi:10.1145/2348283.2348500 dblp:conf/sigir/LogachevS12 fatcat:dfxmnbrhmrcutb4czj3b6hp2n4