Time-aware recommender systems: a comprehensive survey and analysis of existing evaluation protocols

Pedro G. Campos, Fernando Díez, Iván Cantador
<span title="2013-02-15">2013</span> <i title="Springer Nature"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/yrjf5yycp5abxkr5hcdpxjoj5i" style="color: black;">User modeling and user-adapted interaction</a> </i> &nbsp;
Exploiting temporal context has been proved to be an effective approach to improve recommendation performance, as shown, e.g. in the Netflix Prize competition. Time-aware recommender systems (TARS) are indeed receiving increasing attention. A wide range of approaches dealing with the time dimension in user modeling and recommendation strategies have been proposed. In the literature, however, reported results and conclusions about how to incorporate and exploit time information within the
more &raquo; ... ndation processes seem to be contradictory in some cases. Aiming to clarify and address existing discrepancies, in this paper we present a comprehensive survey and analysis of the state of the art on TARS. The analysis show that meaningful divergences appear in the evaluation protocols used-metrics and methodologies. We identify a number of key conditions on offline evaluation of TARS, and based on these conditions, we provide a comprehensive classification of evaluation protocols for TARS. Moreover, we propose a methodological description framework aimed to make the evaluation process fair and reproducible. We also present an empirical study on the impact of different evaluation protocols on measuring relative performances of wellknown TARS. The results obtained show that different uses of the above evaluation 123 68 P. G. Campos et al. conditions yield to remarkably distinct performance and relative ranking values of the recommendation approaches. They reveal the need of clearly stating the evaluation conditions used to ensure comparability and reproducibility of reported results. From our analysis and experiments, we finally conclude with methodological issues a robust evaluation of TARS should take into consideration. Furthermore we provide a number of general guidelines to select proper conditions for evaluating particular TARS.
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1007/s11257-012-9136-x">doi:10.1007/s11257-012-9136-x</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/uygawlf53jh7ndy2lrp4q3lntm">fatcat:uygawlf53jh7ndy2lrp4q3lntm</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20170809123720/http://blog.ag-nbi.de/wp-content/uploads/2015/08/06_TimeAwareRecommender.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/c1/11/c111ad332aaa447fbcdb8c40d4aaa5b5c8e0fbdc.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1007/s11257-012-9136-x"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> springer.com </button> </a>