CENTRE@CLEF2019: Overview of the Replicability and Reproducibility Tasks

Nicola Ferro, Norbert Fuhr, Maria Maistro, Tetsuya Sakai, Ian Soboroff
2019 Conference and Labs of the Evaluation Forum  
Reproducibility has become increasingly important for many research areas, among those IR is not an exception and has started to be concerned with reproducibility and its impact on research results. This paper describes our second attempt to propose a lab on reproducibility named CENTRE, held during CLEF 2019. The aim of CENTRE is to run both a replicability and reproducibility challenge across all the major IR evaluation campaigns and to provide the IR community with a venue where previous
more » ... arch results can be explored and discussed. This paper reports the participant results and preliminary considerations on the second edition of CENTRE@CLEF 2019. Introduction Reproducibility is becoming a primary concern in many areas of science [16, 24] as well as in computer science, as also witnessed by the recent ACM policy on result and artefact review and badging. Also in Information Retrieval (IR) replicability and reproducibility of the experimental results are becoming a more and more central discussion items in the research community [4, 12, 17, 23, 28] . We now commonly find questions about the extent of reproducibility of the reported experiments in the review forms of all the major IR conferences, such as SIGIR, CHIIR, ICTIR and ECIR, as well as journals, such as ACM TOIS. We also witness to the raise of new activities
dblp:conf/clef/0001FMSS19 fatcat:e4l7oosuhvfhtofly45w6s4mau