A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
Filters
Reducing Offline Evaluation Bias in Recommendation Systems
[article]
2014
arXiv
pre-print
Preserved Fulltext
This paper analyses this evaluation bias and proposes a simple item weighting solution that reduces its impact. ...
This adaptation process influences the way users interact with the system and, as a consequence, increases the difficulty of evaluating a recommendation algorithm with historical data (via offline evaluation ...
A strong assumption we make is that in practice reducing offline evaluation bias for constant algorithms contributes to reducing offline evaluation bias for all algorithms. ...
arXiv:1407.0822v1
fatcat:vjrof7qe4jaufa5bml4rrfl5jq
Reducing offline evaluation bias of collaborative filtering algorithms
[article]
2015
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
This paper presents a new application of a weighted offline evaluation to reduce this bias for collaborative filtering algorithms. ...
It thus influences the way users interact with the system and, as a consequence, bias the evaluation of the performance of a recommendation algorithm computed using historical data (via offline evaluation ...
Thus those campaigns have strongly biased the collected data, leading to a significant bias in the offline evaluation score. ...
arXiv:1506.04135v1
fatcat:osqalsytu5gbrdpt5jbjhhs6ai
Recommending Dream Jobs in a Biased Real World
[article]
2019
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
These biases impact the performance of various components of recommender systems, from offline training, to evaluation and online serving of recommendations in production systems. ...
Specific techniques can help reduce bias at each stage of a recommender system. ...
WHY REDUCING BIAS MATTERS Biases impact the performance of various components of recommender systems, from offline training, to evaluation, to online serving of recommendations in production systems. ...
arXiv:1905.06134v1
fatcat:gp4bbyjx4ncrrldhdrih3436bq
Study of a bias in the offline evaluation of a recommendation algorithm
[article]
2015
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
This paper describes this bias and discuss the relevance of a weighted offline evaluation to reduce this bias for different classes of recommendation algorithms. ...
It thus influences the way users interact with the system and, as a consequence, bias the evaluation of the performance of a recommendation algorithm computed using historical data (via offline evaluation ...
Thus those campaigns have strongly biased the collected data, leading to a significant bias in the offline evaluation. ...
arXiv:1511.01280v1
fatcat:s4j3pggganfibk4ninktcssubu
A Model-Based Reinforcement Learning with Adversarial Training for Online Recommendation
2019
Neural Information Processing Systems
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
To reduce bias in the learned model and policy, we use a discriminator to evaluate the quality of generated data and scale the generated rewards. ...
Reinforcement learning is well suited for optimizing policies of recommender systems. ...
Offline evaluation The problems of off-policy learning [22, 25, 26] and offline policy evaluation are generally pervasive and challenging in RL, and in recommender systems in particular. ...
dblp:conf/nips/BaiGW19
fatcat:m5lf3d2t7jcvflv4c5cgidl6su
Comparing Offline and Online Recommender System Evaluations on Long-tail Distributions
2015
ACM Conference on Recommender Systems
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
In this investigation, we conduct a comparison between offline and online accuracy evaluation of different algorithms and settings in a real-world content recommender system. ...
accuracy results in offline and online evaluations. ...
ACKNOWLEDGEMENTS Our thanks to CI&T for supporting the development of Smart Canvas R recommender system evaluation framework and to the ITA for providing the research environment. ...
dblp:conf/recsys/MoreiraSC15
fatcat:etn7rpylt5ggndgxmqm2tzx5by
Model-Based Reinforcement Learning with Adversarial Training for Online Recommendation
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
To reduce bias in the learned model and policy, we use a discriminator to evaluate the quality of generated data and scale the generated rewards. ...
Reinforcement learning is well suited for optimizing policies of recommender systems. ...
Offline evaluation The problems of off-policy learning [22, 25, 26] and offline policy evaluation are generally pervasive and challenging in RL, and in recommender systems in particular. ...
arXiv:1911.03845v3
fatcat:qgonaucopfavnms4cud34j6gry
Estimating Error and Bias in Offline Evaluation Results
2020
Proceedings of the 2020 Conference on Human Information Interaction and Retrieval
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Offline evaluations of recommender systems attempt to estimate users' satisfaction with recommendations using static data from prior user interactions. ...
Substantial breakthroughs in recommendation quality, therefore, will be difficult to assess with existing offline techniques. ...
CONCLUSIONS AND FUTURE WORK We have simulated user preference for items and resulting consumption observations in order to estimate error and bias in the results of offline evaluations of recommender systems ...
doi:10.1145/3343413.3378004
dblp:conf/chiir/TianE20
fatcat:dofm7765ircrzbk5tjyunrr2q4
Offline Evaluation and Optimization for Interactive Systems
2015
Proceedings of the Eighth ACM International Conference on Web Search and Data Mining - WSDM '15
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2015; you can also visit the original URL.
The file type is application/pdf.
evaluation
policy
Data
, ,
⋮
, ,
Biases of Direct Method
• Sampling/selection bias
• From production systems
• Simpson's paradox
• Modeling bias
• Insufficient features to fully represent ...
of a news recommendation system • click lift of a new user feature in ad ranking • reduction of time for user to find a relevant URL on SERP • … Sport
User
Article
Click
Movie
Article
Overall ...
: • Can log randomization seed in and check offline to detect bugs Use standard t-test to detect ≠ ...
doi:10.1145/2684822.2697040
dblp:conf/wsdm/Li15
fatcat:2ap6hcpimfh5xogmdzif6ar6ri
Recommendations as Treatments
2021
The AI Magazine
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
This article explains how these techniques enable unbiased offline evaluation and learning despite biased data, and how they can inform considerations of fairness and equity in recommender systems. ...
In recent years, a new line of research has taken an interventional view of recommender systems, where recommendations are viewed as actions that the system takes to have a desired effect. ...
This trade-off between bias and variance is central to offline evaluation. ...
doi:10.1609/aimag.v42i3.18141
fatcat:hdyi4nadijgp3fpieqojib5pfq
Revisiting offline evaluation for implicit-feedback recommender systems
2019
Proceedings of the 13th ACM Conference on Recommender Systems - RecSys '19
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Recommender systems are typically evaluated in an offline setting. ...
CCS CONCEPTS • Information systems → Recommender systems; Evaluation of retrieval results. ...
The biases present in these datasets pose a significant challenge when the data is used to evaluate other competing algorithms in an offline manner. ...
doi:10.1145/3298689.3347069
dblp:conf/recsys/Jeunen19
fatcat:tlm64i2mbza6hequt4xyrhl4zu
Learning to Rank Research Articles: A Case Study of Collaborative Filtering and Learning to Rank in ScienceDirect
2020
International Workshop on Bibliometric-enhanced Information Retrieval
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
We then describe offline and online evaluation, which are essential for productionizing any recommender. ...
However, by learning from subjective user interactions with the recommender system, our relevance model reversed those trends. ...
Designing offline evaluation raises the question of what the recommender system is designed to do. ...
dblp:conf/birws/KershawPHJ20
fatcat:lhqmte2j3ba7tfax2dws47jmiy
Do Offline Metrics Predict Online Performance in Recommender Systems?
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
In this work we investigate the extent to which offline metrics predict online performance by evaluating eleven recommenders across six controlled simulated environments. ...
Recommender systems operate in an inherently dynamical setting. Past recommendations influence future behavior, including which data points are observed and how user preferences change. ...
Offline Evaluation -Algorithm developers use the datasets from step (1) to evaluate their recommender systems. ...
arXiv:2011.07931v1
fatcat:fre2cuepjzcv5gtnk3ulnblywu
Offline Reinforcement Learning for Mobile Notifications
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Finally, we collect data through online exploration in the production system, train an offline Double Deep Q-Network and launch a successful policy online. ...
We describe a state-marginalized importance sampling policy evaluation approach, which can be used to evaluate the policy offline and tune learning hyperparameters. ...
[13] applied a Policy Gradient learning in YouTube recommender system with Off-policy correction for offline reinforcement learning. Ie et al. ...
arXiv:2202.03867v1
fatcat:v4yibo6htvc6jbwgh7s4f5rhyq
Overview of NewsREEL'16: Multi-dimensional Evaluation of Real-Time Stream-Recommendation Algorithms
[chapter]
2016
Lecture Notes in Computer Science
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
The CLEF News-REEL challenge is a campaign-style evaluation lab allowing participants to tackle news recommendation and to optimize and evaluate their recommender algorithms both online and offline. ...
In the intersection of these perspectives, new insights can be gained on how to effectively evaluate real-time stream recommendation algorithms. ...
The research leading to these results was performed in the Crow-dRec project, which has received funding from the European Union Seventh Framework Program FP7/2007-2013 under grant agreement No. 610594 ...
doi:10.1007/978-3-319-44564-9_27
fatcat:dtmwy2ipj5di7dhywxmd45i5vq
« Previous
Showing results 1 — 15 out of 22,088 results