A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is
Monte Carlo Estimates of Evaluation Metric Error and Bias
Computer Science Faculty Publications and Presentations
Traditional offline evaluations of recommender systems apply metrics from machine learning and information retrieval in settings where their underlying assumptions no longer hold. This results in significant error and bias in measures of top-N recommendation performance, such as precision, recall, and nDCG. Several of the specific causes of these errors, including popularity bias and misclassified decoy items, are well-explored in the existing literature. In this paper we survey a range of workdoi:10.18122/cs_facpubs/148/boisestate fatcat:6epjicqdzff6hmnkq4oxkuxnxa