A Formal Account of Effectiveness Evaluation and Ranking Fusion

Enrique Amigó, Fernando Giner, Stefano Mizzaro, Damiano Spina
2018 Proceedings of the 2018 ACM SIGIR International Conference on Theory of Information Retrieval - ICTIR '18  
This paper proposes a theoretical framework which models the information provided by retrieval systems in terms of Information Theory. The proposed framework allows to formalize: (i) system effectiveness as an information theoretic similarity between system outputs and human assessments, and (ii) ranking fusion as an information quantity measure. As a result, the proposed effectiveness metric improves popular metrics in terms of formal constraints. In addition, our empirical experiments suggest
more » ... that it captures quality aspects from traditional metrics, while the reverse is not true. Our work also advances the understanding of theoretical foundations of the empirically known phenomenon of effectiveness increase when combining retrieval system outputs in an unsupervised manner.
doi:10.1145/3234944.3234958 dblp:conf/ictir/AmigoGMS18 fatcat:2y7yyfkrdrhg7cdhpig6ajauxu