14,315 Hits in 9.0 sec

Stream-Based Recommendations: Online and Offline Evaluation as a Service [chapter]

Benjamin Kille, Andreas Lommatzsch, Roberto Turrin, András Serény, Martha Larson, Torben Brodt, Jonas Seiler, Frank Hopfgartner
2015 Lecture Notes in Computer Science  
The CLEF NewsREEL challenge is a campaign-style evaluation lab allowing participants to evaluate and optimize news recommender algorithms online and offline.  ...  The framework makes possible the reproducible evaluation of recommender algorithms for stream data, taking into account recommender precision as well as the technical complexity of the recommender algorithms  ...  Further, the Idomaar framework for offline evaluation of stream recommendation is a powerful tool that allowing multi-dimensional evaluation of recommender systems "as a service".  ... 
doi:10.1007/978-3-319-24027-5_48 fatcat:vucqtwml3rgtjbmr3x4mwafmqa

Statistically robust evaluation of stream-based recommender systems

Joao Vinagre, Alipio Mario Jorge, Conceicao Rocha, Joao Gama
2019 IEEE Transactions on Knowledge and Data Engineering  
Our results show that besides allowing a real-time, fine-grained online assessment, the online versions of the statistical tests are at least as robust as the batch versions, and definitely more robust  ...  We propose a k-fold validation framework for the pairwise comparison of recommendation algorithms that learn from user feedback streams, using prequential evaluation.  ...  ACKNOWLEDGEMENTS This work is financed by National Funds through the Portuguese funding agency, FCT -Fundac ¸ão para a Ciência e a Tecnologia within project: UID/EEA/50014/2019.  ... 
doi:10.1109/tkde.2019.2960216 fatcat:bc2vmucmgfcpndcgejkfbo2xfy

A Stream-based Resource for Multi-Dimensional Evaluation of Recommender Algorithms

Benjamin Kille, Andreas Lommatzsch, Frank Hopfgartner, Martha Larson, Arjen P. de Vries
2017 Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval - SIGIR '17  
The data set allows researchers to study a stream recommendation problem closely by "replaying" it locally, and ORP makes it possible to take this evaluation "live" in a living lab scenario.  ...  We introduce two resources supporting such evaluation methodologies: the new data set of stream recommendation interactions released for CLEF NewsREEL 2017, and the new Open Recommendation Platform (ORP  ...  Discussion Both online and offline resources are inherently based on stream data.  ... 
doi:10.1145/3077136.3080726 dblp:conf/sigir/KilleLHLV17 fatcat:ldua3jrhgzfv5cximbxxhkeww4

CLEF 2017 NewsREEL Overview: Offline and Online Evaluation of Stream-based News Recommender Systems

Benjamin Kille, Andreas Lommatzsch, Frank Hopfgartner, Martha A. Larson, Torben Brodt
2017 Conference and Labs of the Evaluation Forum  
The CLEF NewsREEL challenge allows researchers to evaluate news recommendation algorithms both online (NewsREEL Live) and offline (News-REEL Replay).  ...  Compared with the previous year NewsREEL challenged participants with a higher volume of messages and new news portals.  ...  Introduction The development of recommender services based on stream data is a challenging task.  ... 
dblp:conf/clef/KilleLHLB17 fatcat:k5aelrasrrdr7cve3u5hszgsfq

CLEF 2017 NewsREEL Overview: A Stream-Based Recommender Task for Evaluation and Education [chapter]

Andreas Lommatzsch, Benjamin Kille, Frank Hopfgartner, Martha Larson, Torben Brodt, Jonas Seiler, Özlem Özgöbek
2017 Lecture Notes in Computer Science  
As other online, stream-based recommender systems, they face particular challenges, including limited information on users' preferences and also rapidly fluctuating item collections.  ...  NewsREEL represents a unique opportunity to evaluate recommendation algorithms and for students to experience realistic conditions and to enlarge their skill sets.  ...  The NewsREEL Live task offers a living lab environment in which participants can test their stream-based recommendation algorithms online.  ... 
doi:10.1007/978-3-319-65813-1_23 fatcat:mklyne35mjg63hfnhnm6sry2ai

Benchmarking News Recommendations

Frank Hopfgartner, Torben Brodt, Jonas Seiler, Benjamin Kille, Andreas Lommatzsch, Martha Larson, Roberto Turrin, András Serény
2016 SIGIR Forum  
The CLEF NewsREEL challenge is a campaign-style evaluation lab allowing participants to evaluate and optimize news recommender algorithms.  ...  The lab challenges participants to compete in either a "living lab" (Task 1) or perform an evaluation that replays recorded streams (Task 2).  ...  The offline evaluation (based on a data set recorded between July and August 2015) enabled the reproducible evaluation of stream-based recommender algorithms.  ... 
doi:10.1145/2888422.2888443 fatcat:rgyv32ohmbh6rp2yypjygqohda

New fair QoS-based charging solution for mobile multimedia streams

Zsolt Butyka, Tamas Jursonovics, Sandor Imre
2008 International Journal of Virtual Technology and Multimedia  
The algorithms are evaluated on a real GPRS streaming flow.  ...  In this paper, we present a new streaming proxy based charging architecture for QoS differentiated charging in 3G.  ...  Acknowledgements This work is supported by ETIK and OTKA F042590.  ... 
doi:10.1504/ijvtm.2008.017107 fatcat:itqnzx5xzrbdhhm24vkhystt4e

Real-time Recommendation of Streamed Data

Frank Hopfgartner, Benjamin Kille, Tobias Heintz, Roberto Turrin
2015 Proceedings of the 9th ACM Conference on Recommender Systems  
Focusing on the news domain, participants learned how to benchmark the performance of stream-based recommendation algorithms in a live recommender system and in a simulated environment.  ...  This tutorial addressed two trending topics in the field of recommender systems research, namely A/B testing and realtime recommendations of streamed data.  ...  An alternative evaluation paradigm is online evaluation, also referred to as A/B testing [1] . Online evaluation aims to benchmark varieties of a recommender system by a larger group of users.  ... 
doi:10.1145/2792838.2792839 fatcat:yqmphirohzaylmodouuiwzcm54

Idomaar: A Framework for Multi-dimensional Benchmarking of Recommender Algorithms

Mario Scriminaci, Andreas Lommatzsch, Benjamin Kille, Frank Hopfgartner, Martha A. Larson, Davide Malagoli, András Serény, Till Plumbaum
2016 ACM Conference on Recommender Systems  
Idomaar goes beyond current academic research practices by creating a realistic evaluation environment and computing both effectiveness and technical metrics for stream-based as well as setbased evaluation  ...  In real-world scenarios, recommenders face non-functional requirements of technical nature and must handle dynamic data in the form of sequential streams.  ...  The framework uses large-scale static data sets to simulate live data streams, bringing offline evaluation closer to online A/B testing.  ... 
dblp:conf/recsys/ScriminaciLKHLM16 fatcat:hd52pzoqbbd4hieg2rk64oryne

Recommendations as Treatments

Thorsten Joachims, Ben London, Yi Su, Adith Swaminathan, Lequn Wang
2021 The AI Magazine  
This article explains how these techniques enable unbiased offline evaluation and learning despite biased data, and how they can inform considerations of fairness and equity in recommender systems.  ...  In recent years, a new line of research has taken an interventional view of recommender systems, where recommendations are viewed as actions that the system takes to have a desired effect.  ...  To make this issue more concrete, consider a simple movie recommender for a video streaming service.  ... 
doi:10.1609/aimag.v42i3.18141 fatcat:hdyi4nadijgp3fpieqojib5pfq

Algorithms and System Architecture for Immediate Personalized News Recommendations [article]

Takeshi Yoneda, Shunsuke Kozawa, Keisuke Osone, Yukinori Koide, Yosuke Abe, Yoshifumi Seki
2019 arXiv   pre-print
We evaluate the proposed method both offline and online.  ...  The offline experiments are conducted through a real-world dataset from a commercial news delivery service, and online experiments are conducted via A/B testing on production environments.  ...  ACKNOWLEDGMENTS We would like to thank the Engineering Teams of our news delivery services, the Machine Learning Team, and the Data Management Platform Team for their contributions to this project.  ... 
arXiv:1909.01005v1 fatcat:qnhbxxabrzd3ne4v4wrx7ujpku

Real-time top-n recommendation in social streams

Ernesto Diaz-Aviles, Lucas Drumond, Lars Schmidt-Thieme, Wolfgang Nejdl
2012 Proceedings of the sixth ACM conference on Recommender systems - RecSys '12  
We consider collaborative filtering as an online ranking problem and present Stream Ranking Matrix Factorization -RMFX -, which uses a pairwise approach to matrix factorization in order to optimize the  ...  People generate and consume data in real-time within social networking services, such as Twitter, and increasingly rely upon continuous streams of messages for real-time access to fresh knowledge about  ...  Recommendation Quality We report in this section the CPU training times and space required for the best performing variation of our online approach: RMFX, and the ones for the strongest baseline: WRMF.  ... 
doi:10.1145/2365952.2365968 dblp:conf/recsys/Diaz-AvilesDSN12 fatcat:pzemwfmflndn5iqe245p6mwpru

A Survey of Recommendation Systems: Recommendation Models, Techniques, and Application Fields

Hyeyoung Ko, Suyeon Lee, Yoonseo Park, Anna Choi
2022 Electronics  
various recommendation system studies, as well as applied service field industry data.  ...  As a result of this study, it was found that the flow and quantitative growth of various detailed studies of recommendation systems interact with the business growth of the actual applied service field  ...  Product recommendation system that extends item-based CF through online-offline data combination. Figure 19 . 19 Figure 19.  ... 
doi:10.3390/electronics11010141 fatcat:k2tpqmtufrf57kgquuyhzae3da

Insights on Social Recommender Systems

Wolney Leal De Mello Neto, Ann Nowé
2012 ACM Conference on Recommender Systems  
Assuming a user has a dense social network, the cold-start problem can be easily tackled. Finally, rating prediction accuracy performs better when evaluated online than by offline cross-validation.  ...  Results from traditional evaluation by offline cross-validation are compared to measuring prediction accuracy of online feedback data.  ...  This research is part of a master studies sponsored by Monesia: MObility Network Europe-Southamerica: an Institutional Approach, an Erasmus Mundus External Cooperation Window.  ... 
dblp:conf/recsys/NetoN12 fatcat:zp2xsbm7hjcvrelc532us7m3om

Revisiting offline evaluation for implicit-feedback recommender systems

Olivier Jeunen
2019 Proceedings of the 13th ACM Conference on Recommender Systems - RecSys '19  
Recommender systems are typically evaluated in an offline setting.  ...  Online evaluation is effective, but inefficient for a number of reasons. Offline evaluation is much more efficient, but current methodologies often fail to accurately predict online performance.  ...  RecSys '19, behaviour on a music streaming service, video watches on video streaming websites, and many more.  ... 
doi:10.1145/3298689.3347069 dblp:conf/recsys/Jeunen19 fatcat:tlm64i2mbza6hequt4xyrhl4zu
« Previous Showing results 1 — 15 out of 14,315 results