48 Hits in 4.6 sec

Benchmarking News Recommendations

Frank Hopfgartner, Torben Brodt, Jonas Seiler, Benjamin Kille, Andreas Lommatzsch, Martha Larson, Roberto Turrin, András Serény
2016 SIGIR Forum  
The CLEF NewsREEL challenge is a campaign-style evaluation lab allowing participants to evaluate and optimize news recommender algorithms.  ...  In this report, we discuss the objectives and challenges of the NewsREEL lab, summarize last year's campaign and outline the main research challenges that can be addressed by participating in NewsREEL  ...  Acknowledgment The work leading to these results has received funding (or partial funding) from the European Union's Seventh Framework Programme (FP7/2007-2013) under grant agreement number 610594 (CrowdRec  ... 
doi:10.1145/2888422.2888443 fatcat:rgyv32ohmbh6rp2yypjygqohda

Benchmarking News Recommendations in a Living Lab [chapter]

Frank Hopfgartner, Benjamin Kille, Andreas Lommatzsch, Till Plumbaum, Torben Brodt, Tobias Heintz
2014 Lecture Notes in Computer Science  
The living lab has first been organized as News Recommendation Challenge at ACM RecSys'13 and then as campaign-style evaluation lab NEWSREEL at CLEF'14.  ...  We outline the living lab scenario and the experimental setup of the two benchmarking events.  ...  Acknowledgement The work leading to these results has received funding (or partial funding) from the Central Innovation Programme for SMEs of the German Federal Ministry for Economic Affairs and Energy  ... 
doi:10.1007/978-3-319-11382-1_21 fatcat:2a25ii4wpbeldb27jnax5ewbzq

Keynote: Capturing User Interests for Content-based Recommendations

Frank Hopfgartner
2015 ACM Conference on Recommender Systems  
Focusing on the latter, this keynote presents various examples and case studies that illustrate both strengths and weaknesses of content-based recommendation.  ...  referred to as content-based recommendation.  ...  The talk ends with an overview of NewsREEL 1 , an evaluation campaign that allows researchers to benchmark news article recommendation algorithms in an offline [9] and an online [1, 5] setting.  ... 
dblp:conf/recsys/Hopfgartner15 fatcat:7jhvlu7mqfccnfu5sxskkyn65m

Real-time Recommendation of Streamed Data

Frank Hopfgartner, Benjamin Kille, Tobias Heintz, Roberto Turrin
2015 Proceedings of the 9th ACM Conference on Recommender Systems  
Focusing on the news domain, participants learned how to benchmark the performance of stream-based recommendation algorithms in a live recommender system and in a simulated environment.  ...  This tutorial addressed two trending topics in the field of recommender systems research, namely A/B testing and realtime recommendations of streamed data.  ...  This recommender scenario is realized in the News REcommendation Evaluation Lab (NewsREEL) 1 , a campaign style evaluation lab that focuses on benchmarking stream-based recommenders.  ... 
doi:10.1145/2792838.2792839 fatcat:yqmphirohzaylmodouuiwzcm54

Join the Living Lab: Evaluating News Recommendations in Real-Time [chapter]

Frank Hopfgartner, Torben Brodt
2015 Lecture Notes in Computer Science  
Participants of this tutorial learnt how to participate in CLEF NEWSREEL, a living lab for the evaluation of news recommender algorithms.  ...  Various research challenges can be addressed within NEWS-REEL, such as the development and evaluation of collaborative filtering or content-based filtering strategies.  ...  In the remainder of this section, we briefly outline the new recommendation use case.  ... 
doi:10.1007/978-3-319-16354-3_95 fatcat:joeelkeqmzfivnqokrqg2ytstu

CLEF 2017 NewsREEL Overview: A Stream-Based Recommender Task for Evaluation and Education [chapter]

Andreas Lommatzsch, Benjamin Kille, Frank Hopfgartner, Martha Larson, Torben Brodt, Jonas Seiler, Özlem Özgöbek
2017 Lecture Notes in Computer Science  
News recommender systems provide users with access to news stories that they find interesting and relevant.  ...  This paper presents NewsREEL 2017 and also provides insights into the effectiveness of NewsREEL to support the goals of instructors teaching recommender systems to students.  ...  [9] present the case of a Swiss news publisher. They devise a system using context trees to capture changing preferences.  ... 
doi:10.1007/978-3-319-65813-1_23 fatcat:mklyne35mjg63hfnhnm6sry2ai

CLEF NewsREEL 2016: Comparing Multi-dimensional Offline and Online Evaluation of News Recommender Systems

Benjamin Kille, Andreas Lommatzsch, Frank Hopfgartner, Martha A. Larson, Jonas Seiler, Davide Malagoli, András Serény, Torben Brodt
2016 Conference and Labs of the Evaluation Forum  
Running in its third year at CLEF, NewsREEL challenged participants to develop news recommendation algorithms and have them benchmarked in an online (Task 1) and offline setting (Task 2), respectively.  ...  This paper provides an overview of the NewsREEL scenario, outlines this year's campaign, presents results of both tasks, and discusses the approaches of participating teams.  ...  The "New Ideas" track was jointly organized with the organizers of CLEF LL4IR.  ... 
dblp:conf/clef/KilleLHLSMSB16 fatcat:ae6zwyy56vey3at3qhavtetry4

Stream-Based Recommendations: Online and Offline Evaluation as a Service [chapter]

Benjamin Kille, Andreas Lommatzsch, Roberto Turrin, András Serény, Martha Larson, Torben Brodt, Jonas Seiler, Frank Hopfgartner
2015 Lecture Notes in Computer Science  
In this paper, we discuss the objectives and challenges of the NewsREEL lab. We motivate the metrics used for benchmarking the recommender algorithms and explain the challenge dataset.  ...  The CLEF NewsREEL challenge is a campaign-style evaluation lab allowing participants to evaluate and optimize news recommender algorithms online and offline.  ...  Acknowledgments The research leading to these results was performed in the CrowdRec project, which has received funding from the European Union Seventh Framework Program FP7/2007-2013 under grant agreement  ... 
doi:10.1007/978-3-319-24027-5_48 fatcat:vucqtwml3rgtjbmr3x4mwafmqa

Recommender Systems Evaluations : Offline, Online, Time and A/A Test

Gebrekirstos G. Gebremeskel, Arjen P. de Vries
2016 Conference and Labs of the Evaluation Forum  
The fourth dimension is the quantification of the effect of non-algorithmic factors on the performance of an online recommender system by using an A/A test.  ...  We present a comparison of recommender systems algorithms along four dimensions. The first dimension is offline evaluation where we compare the performance of our algorithms in an offline setting.  ...  In 2015, we participated in CLEF NewsREEL News Recommendations Evaluation, the task of Benchmark News Recommendations in a Living Lab [6] .  ... 
dblp:conf/clef/GebremeskelV16 fatcat:li3k6wur4vdgtdzg7qngr3eq3a

Development and Evaluation of a Highly Scalable News Recommender System

Ilya Verbitskiy, Patrick Probst, Andreas Lommatzsch
2015 Conference and Labs of the Evaluation Forum  
In this paper we describe our participation at the CLEF-NewsREEL challenge 2015. We present our highly scalable implementation of a news recommendation algorithm.  ...  The evaluation shows that our system implemented using the Akka framework scales well with the restrictions and outperforms the recommendation precision of the baseline recommender.  ...  Acknowledgement This research is supported by funding from the European Commission's 7th Framework Program (FP7/2007-2013) under grant agreement number 610594.  ... 
dblp:conf/clef/VerbitskiyPL15 fatcat:6pojyqd43rce7bd5wtsfongk3y

CLEF NewsREEL 2016: Image based Recommendation

Francesco Corsini, Martha A. Larson
2016 Conference and Labs of the Evaluation Forum  
Our approach to the CLEF NewsREEL 2016 News Recommendation Evaluation Lab investigates the connection between images and users clicking behavior.  ...  We experiment with visual information, namely Face Detection and Saliency Map, extracted from the images that accompany news items to see if they can be used to chose news items that have a higher chance  ...  More experiments with different baseline combinations and settings are required in the future to definitively prove the effectiveness of image-based recommendation in the news environment.  ... 
dblp:conf/clef/CorsiniL16 fatcat:jr4gh5h35baerd6vr5j6qvzkk4

A Stream-based Resource for Multi-Dimensional Evaluation of Recommender Algorithms

Benjamin Kille, Andreas Lommatzsch, Frank Hopfgartner, Martha Larson, Arjen P. de Vries
2017 Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval - SIGIR '17  
We introduce two resources supporting such evaluation methodologies: the new data set of stream recommendation interactions released for CLEF NewsREEL 2017, and the new Open Recommendation Platform (ORP  ...  To our knowledge, NewsREEL is the first online news recommender system resource to be put at the disposal of the research community.  ...  The resources are released by REcommendation Evaluation Lab (NewsREEL) [5] , a campaign-style evaluation lab of the CLEF conference.  ... 
doi:10.1145/3077136.3080726 dblp:conf/sigir/KilleLHLV17 fatcat:ldua3jrhgzfv5cximbxxhkeww4

A System for Online News Recommendations in Real-Time with Apache Mahout

Paul David Beck, Manuel Blaser, Adrian Michalke, Andreas Lommatzsch
2017 Conference and Labs of the Evaluation Forum  
Two algorithms are combined to ensure highly precise recommendations and a high reliability. The system is evaluated in the CLEF NEWSREEL challenge.  ...  In this work, we present our recommender system built based on APACHE MA-HOUT tailored to the needs of news recommender systems.  ...  Problem Description In this work, we analyze the news recommendation task defined in the CLEF NEWS-REEL challenge [9] .  ... 
dblp:conf/clef/BeckBML17 fatcat:az5jlhwxzzf47kbd6d3rjbd4ni

Evaluation Infrastructures for Academic Shared Tasks

Johann Schaible, Timo Breuer, Narges Tavakolpoursaleh, Bernd Müller, Benjamin Wolff, Philipp Schaer
2020 Datenbank-Spektrum  
In this paper, we elaborate on the benefits and shortcomings of four state-of-the-art evaluation infrastructures on search and recommendation tasks concerning the following requirements: support for online  ...  In addition, we introduce an evaluation infrastructure concept design aiming at reducing the shortcomings in shared tasks for search and recommender systems.  ...  Acknowledgements The STELLA project is funded by the Deutsche Forschungsgemeinschaft (DFG) -Project number 407518790.  ... 
doi:10.1007/s13222-020-00335-x fatcat:f7lj6fr4n5hazoqtywrulw7b4u

News Recommender System: A review of recent progress, challenges, and opportunities [article]

Shaina Raza, Chen Ding
2021 arXiv   pre-print
In the first part, we present an overview of the conventional recommendation solutions, datasets, evaluation criteria beyond accuracy and recommendation platforms being used in NRS.  ...  Due to the rapid growth of building recommender systems using deep learning models, we divide our discussion in two parts.  ...  CLEF NEWSREEL and Open Recommendation Platform (ORP), CLEF NEWSREEL platform was designed to encourage researchers to develop novel recommenders using the Plista dataset and evaluate them in real time  ... 
arXiv:2009.04964v4 fatcat:s7jl63nwm5e55myezsxpzquuje
« Previous Showing results 1 — 15 out of 48 results