Filters








137 Hits in 4.8 sec

CLEF 2017 Dynamic Search Evaluation Lab Overview [chapter]

Evangelos Kanoulas, Leif Azzopardi
2017 Lecture Notes in Computer Science  
In this paper we provide an overview of the first edition of the CLEF Dynamic Search Lab.  ...  Herein, we describe the overall objectives of the CLEF 2017 Dynamic Search Lab, the resources created for the pilot task, the evaluation methodology adopted, and some preliminary evaluation results of  ...  Conclusions The goal of the CLEF 2017 Dynamic Search lab is to answer one key question: how can we evaluate dynamic search algorithms?  ... 
doi:10.1007/978-3-319-65813-1_31 fatcat:bczfn7xtvbgivgyppwm7ka7xxy

Overview of the CLEF Dynamic Search Evaluation Lab 2018 [chapter]

Evangelos Kanoulas, Leif Azzopardi, Grace Hui Yang
2018 Lecture Notes in Computer Science  
In this paper we provide an overview of the CLEF 2018 Dynamic Search Lab. The lab ran for the first time in 2017 as a workshop.  ...  Herein, we describe the overall objectives of the CLEF 2018 Dynamic Search Lab, the resources created, and the evaluation methodology designed.  ...  The CLEF 2018 Dynamic Search Lab focuses on the development of a dynamic search system evaluation framework, on the basis of the conclusions of the CLEF 2018 Dynamic Search Lab workshop [7, 8] .  ... 
doi:10.1007/978-3-319-98932-7_31 fatcat:hbwbpbngvfbexoif34prsout4i

Leveraging Wikipedia's Article Structure to Build Search Agents: TUW at CLEF 2017 Dynamic Search

João R. M. Palotti
2017 Conference and Labs of the Evaluation Forum  
Dynamic Search [2] .  ...  The Wikipedia article Smoking Cessation 1 is the top hit for such query Fig. 1 : Five runs were submitted at CLEF Dynamic Search: one simple baseline made up using only the initial query and four runs  ... 
dblp:conf/clef/Palotti17 fatcat:65wpl7zxnzco3of3ebry3naupi

A Stream-based Resource for Multi-Dimensional Evaluation of Recommender Algorithms

Benjamin Kille, Andreas Lommatzsch, Frank Hopfgartner, Martha Larson, Arjen P. de Vries
2017 Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval - SIGIR '17  
We introduce two resources supporting such evaluation methodologies: the new data set of stream recommendation interactions released for CLEF NewsREEL 2017, and the new Open Recommendation Platform (ORP  ...  The data set allows researchers to study a stream recommendation problem closely by "replaying" it locally, and ORP makes it possible to take this evaluation "live" in a living lab scenario.  ...  NewsREEL is designed to unite both worlds by providing resources that allow both online and offline evaluation. Figure 1 : 1 Figure 1: CLEF NewsREEL 2017 overview.  ... 
doi:10.1145/3077136.3080726 dblp:conf/sigir/KilleLHLV17 fatcat:ldua3jrhgzfv5cximbxxhkeww4

CLEF 2017 NewsREEL Overview: A Stream-Based Recommender Task for Evaluation and Education [chapter]

Andreas Lommatzsch, Benjamin Kille, Frank Hopfgartner, Martha Larson, Torben Brodt, Jonas Seiler, Özlem Özgöbek
2017 Lecture Notes in Computer Science  
NewsREEL represents a unique opportunity to evaluate recommendation algorithms and for students to experience realistic conditions and to enlarge their skill sets.  ...  The NewsREEL Challenge requires participants to conduct data-driven experiments in NewsREEL Replay as well as deploy their best models into NewsREEL Live's 'living lab'.  ...  Participants in the CLEF NewsREEL (News REcommendation Evaluation Lab) challenge face a complex environment. They have to define a strategy to produce accurate recommendations.  ... 
doi:10.1007/978-3-319-65813-1_23 fatcat:mklyne35mjg63hfnhnm6sry2ai

A News Recommender Engine with a Killer Sequence

Pieter Bons, Nick Evans, Peter Kampstra, Timo van Kessel
2017 Conference and Labs of the Evaluation Forum  
Our submission to the CLEF NewsREEL 2107 News Recommendation Evaluation Lab attempts to apply the concept of storytelling to the participating news domains.  ...  For articles that were not yet part of any sequence we generated the recommendations from a dynamic set of the most recently changed publications for that domain.  ...  Approach The Living Lab Evaluation [2] was performed using the open recommendation platform (ORP) provided by plista.  ... 
dblp:conf/clef/BonsEKK17 fatcat:x2mmaja2lbhnrotaxyo5gja3am

Ranking and Feedback-based Stopping for Recall-Centric Document Retrieval

Noah Hollmann, Carsten Eickhoff
2017 Conference and Labs of the Evaluation Forum  
This paper describes ETH Zurich's participation in "Task 2: Technologically Assisted Reviews in Empirical Medicine" at the CLEF eHealth Evaluation lab 2017 [7] .  ...  We introduce a parameter β that defines the sampling rate β = Experiments Data set The experiments are evaluated on the data set provided by the CLEF 2017 Task 2 eHealth Challenge [10, 7] .  ... 
dblp:conf/clef/HollmannE17 fatcat:xm5yhsbcszcnjmtl6rvzelc57a

Evaluation-as-a-Service for the Computational Sciences

Frank Hopfgartner, Noriko Kando, Makoto P. Kato, Anastasia Krithara, Tim Gollub, Martin Potthast, Evelyne Viegas, Simon Mercer, Allan Hanbury, Henning Müller, Ivan Eggel, Krisztian Balog (+4 others)
2018 Journal of Data and Information Quality  
Several research domains such as information retrieval have long relied on systematic evaluation to measure progress: here, the Cranfield paradigm of creating shared test collections, defining search tasks  ...  This paper is based on discussions at a workshop on Evaluation-as-a-Service (EaaS).  ...  2014 -2017 as a campaign-style evaluation lab at CLEF.  ... 
doi:10.1145/3239570 dblp:journals/jdiq/HopfgartnerHMEB18 fatcat:6pe7jqqkh5hi3d3wmdly4gb5vi

WBI at CLEF eHealth 2018 Task 1: Language-independent ICD-10 Coding using Multi-lingual Embeddings and Recurrent Neural Networks

Jurica Seva, Mario Sänger, Ulf Leser
2018 Conference and Labs of the Evaluation Forum  
This paper describes the participation of the WBI team in the CLEF eHealth 2018 shared task 1 ("Multilingual Information Extraction -ICD-10 coding").  ...  During evaluation our best model achieves an F-score of 0.34 for French, 0.45 for Hungarian and 0.77 for Italian.  ...  ICD-10 Classification The ICD-10 coding task has already been carried out in the 2016 [26] and 2017 [25] edition of the eHealth lab.  ... 
dblp:conf/clef/SevaSL18 fatcat:vjrfo3ugjnaqll5ztivpgd3phi

Annotation of Clinical Narratives in Bulgarian language

Ivaylo Radev, Kiril Simov, Galia Angelova, Svetla Boytcheva
2017 Proceedings of the Biomedical NLP Workshop  
The annotated corpus will be used as a Gold standard for information extraction evaluation of test corpus of 6,200 discharge letters.  ...  Acknowledgments The research presented here is partially supported by the grant SpecialIZed Data MIning MethoDs Based on Semantic Attributes (IZIDA), funded by the Bulgarian National Science Fund in 2017  ...  The annotated corpus will be used for information extraction evaluation. The paper is structured as follows. Section 2 overviews related work with focus on the technological solutions.  ... 
doi:10.26615/978-954-452-044-1_011 dblp:conf/ranlp/RadevSAB17 fatcat:bqbgov72ijch7cxt7gpae4kxpe

News Recommender System: A review of recent progress, challenges, and opportunities [article]

Shaina Raza, Chen Ding
2021 arXiv   pre-print
In the first part, we present an overview of the conventional recommendation solutions, datasets, evaluation criteria beyond accuracy and recommendation platforms being used in NRS.  ...  The researchers used CLEF NEWSREEL for online evaluation as well as the replay-based (simulation or offline) evaluation (Domann and Lommatzsch 2017; Kumar et al. 2017) .  ...  The CLEF NEWSREEL platform (a campaign-style evaluation lab) was designed to encourage researchers to develop novel recommenders for news domains, so we see a clear rise in the number of publications during  ... 
arXiv:2009.04964v4 fatcat:s7jl63nwm5e55myezsxpzquuje

2dSearch: A Visual Approach to Search Strategy Formulation

Tony Russell-Rose, Phil Gooch
2018 Biennial Conference on Design of Experimental Search & Information Retrieval Systems  
Automated search term suggestions are provided using a combination of knowledge-based and statistical natural language processing techniques.  ...  Knowledge workers (such as healthcare information professionals, patent agents and media monitoring professionals) need to create and execute search strategies that are accurate, repeatable and transparent  ...  ACKNOWLEDGMENTS Part of this work was funded by Technology Strategy Board grant 131641 "A visual framework for search query formulation" and Innovate UK grant 102975 "Intelligent search assistance".  ... 
dblp:conf/desires/Russell-RoseG18 fatcat:sumk6mf7ybahdbov2cpegtw25i

Hierarchical Deep Multi-modal Network for Medical Visual Question Answering [article]

Deepak Gupta, Swati Suman, Asif Ekbal
2020 arXiv   pre-print
We evaluate the performance of our proposed model on two benchmark datasets, viz. RAD and CLEF18.  ...  We follow the evaluation setup discussed in ImageCLEF2018 VQA-Med 2018 challenge overview paper to evaluate the performance of the system.  ...  We follow the evaluation setup discussed in ImageCLEF2018 VQA-Med 2018 challenge overview paper to evaluate the performance of the system.  ... 
arXiv:2009.12770v1 fatcat:d2dmtduat5b3bm4ujgyryh474y

Report on the Information Retrieval Festival (IRFest2017)

Leif Azzopardi, Craig Macdonald, Iadh Ounis, Martin Halvey
2017 SIGIR Forum  
The Information Retrieval Festival took place in April 2017 in Glasgow.  ...  The workshop was followed by a "Tour de Scotland" where delegates were taken from Glasgow to Aberdeen for the European Conference in Information Retrieval (ECIR 2017).  ...  Vol. 51 No. 1 June 2017  ... 
doi:10.1145/3130332.3130336 fatcat:3ejhaputzve5dileftpldba4uu

Question Answering in the Biomedical Domain

Vincent Nguyen
2019 Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop  
In Conference and Labs of the Evaluation Forum, Sheffield, UK. Bryan McCann, Nitish Shirish Keskar, Caiming Xiong, and Richard Socher. 2018.  ...  In order to accommodate for these relationships, we can draw on prior research as starting points such as space-time local embeddings (Sun et al., 2015) , dynamic word embeddings (Bamler and Mandt, 2017  ... 
doi:10.18653/v1/p19-2008 dblp:conf/acl/Nguyen19 fatcat:23yy6eohofh7tepzp4iixore2e
« Previous Showing results 1 — 15 out of 137 results