137 Hits in 5.1 sec

CLEF 2017 Dynamic Search Evaluation Lab Overview [chapter]

Evangelos Kanoulas, Leif Azzopardi
2017 Lecture Notes in Computer Science  
In this paper we provide an overview of the first edition of the CLEF Dynamic Search Lab.  ...  Herein, we describe the overall objectives of the CLEF 2017 Dynamic Search Lab, the resources created for the pilot task, the evaluation methodology adopted, and some preliminary evaluation results of  ...  Acknowledgements This work was partially supported by the Google Faculty Research Award program and the Microsoft Azure for Research Award program (CRM:0518163).  ... 
doi:10.1007/978-3-319-65813-1_31 fatcat:bczfn7xtvbgivgyppwm7ka7xxy

Overview of the CLEF Dynamic Search Evaluation Lab 2018 [chapter]

Evangelos Kanoulas, Leif Azzopardi, Grace Hui Yang
2018 Lecture Notes in Computer Science  
In this paper we provide an overview of the CLEF 2018 Dynamic Search Lab. The lab ran for the first time in 2017 as a workshop.  ...  Herein, we describe the overall objectives of the CLEF 2018 Dynamic Search Lab, the resources created, and the evaluation methodology designed.  ...  All content represents the opinion of the authors, which is not necessarily shared or endorsed by their respective employers and/or sponsors.  ... 
doi:10.1007/978-3-319-98932-7_31 fatcat:hbwbpbngvfbexoif34prsout4i

Leveraging Wikipedia's Article Structure to Build Search Agents: TUW at CLEF 2017 Dynamic Search

João R. M. Palotti
2017 Conference and Labs of the Evaluation Forum  
of such challenges, the first CLEF Dynamic Search [2] .  ...  The Wikipedia article Smoking Cessation 1 is the top hit for such query Fig. 1 : Five runs were submitted at CLEF Dynamic Search: one simple baseline made up using only the initial query and four runs  ... 
dblp:conf/clef/Palotti17 fatcat:65wpl7zxnzco3of3ebry3naupi

A Stream-based Resource for Multi-Dimensional Evaluation of Recommender Algorithms

Benjamin Kille, Andreas Lommatzsch, Frank Hopfgartner, Martha Larson, Arjen P. de Vries
2017 Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval - SIGIR '17  
We introduce two resources supporting such evaluation methodologies: the new data set of stream recommendation interactions released for CLEF NewsREEL 2017, and the new Open Recommendation Platform (ORP  ...  The data set allows researchers to study a stream recommendation problem closely by "replaying" it locally, and ORP makes it possible to take this evaluation "live" in a living lab scenario.  ...  NewsREEL is designed to unite both worlds by providing resources that allow both online and offline evaluation. Figure 1 : 1 Figure 1: CLEF NewsREEL 2017 overview.  ... 
doi:10.1145/3077136.3080726 dblp:conf/sigir/KilleLHLV17 fatcat:ldua3jrhgzfv5cximbxxhkeww4

CLEF 2017 NewsREEL Overview: A Stream-Based Recommender Task for Evaluation and Education [chapter]

Andreas Lommatzsch, Benjamin Kille, Frank Hopfgartner, Martha Larson, Torben Brodt, Jonas Seiler, Özlem Özgöbek
2017 Lecture Notes in Computer Science  
NewsREEL represents a unique opportunity to evaluate recommendation algorithms and for students to experience realistic conditions and to enlarge their skill sets.  ...  This paper presents NewsREEL 2017 and also provides insights into the effectiveness of NewsREEL to support the goals of instructors teaching recommender systems to students.  ...  In News-REEL, our focus is on the static environments of offline experimentation and dynamic environments of online evaluation.  ... 
doi:10.1007/978-3-319-65813-1_23 fatcat:mklyne35mjg63hfnhnm6sry2ai

A News Recommender Engine with a Killer Sequence

Pieter Bons, Nick Evans, Peter Kampstra, Timo van Kessel
2017 Conference and Labs of the Evaluation Forum  
Our submission to the CLEF NewsREEL 2107 News Recommendation Evaluation Lab attempts to apply the concept of storytelling to the participating news domains.  ...  For articles that were not yet part of any sequence we generated the recommendations from a dynamic set of the most recently changed publications for that domain.  ...  Approach The Living Lab Evaluation [2] was performed using the open recommendation platform (ORP) provided by plista.  ... 
dblp:conf/clef/BonsEKK17 fatcat:x2mmaja2lbhnrotaxyo5gja3am

Ranking and Feedback-based Stopping for Recall-Centric Document Retrieval

Noah Hollmann, Carsten Eickhoff
2017 Conference and Labs of the Evaluation Forum  
Using 20 training and 30 test topics, we achieve a mean recall of 93.3%, filtering out 59.1% of the articles.  ...  The method is especially suited for scenarios with sufficiently many relevant articles (>5) that can be sampled and employed for relevance feedback.  ...  This paper describes ETH Zurich's participation in "Task 2: Technologically Assisted Reviews in Empirical Medicine" at the CLEF eHealth Evaluation lab 2017 [7] .  ... 
dblp:conf/clef/HollmannE17 fatcat:xm5yhsbcszcnjmtl6rvzelc57a

Evaluation-as-a-Service for the Computational Sciences

Frank Hopfgartner, Noriko Kando, Makoto P. Kato, Anastasia Krithara, Tim Gollub, Martin Potthast, Evelyne Viegas, Simon Mercer, Allan Hanbury, Henning Müller, Ivan Eggel, Krisztian Balog (+4 others)
2018 Journal of Data and Information Quality  
Several research domains such as information retrieval have long relied on systematic evaluation to measure progress: here, the Cranfield paradigm of creating shared test collections, defining search tasks  ...  Evaluation in empirical computer science is essential to show progress and assess technologies developed.  ...  2014 -2017 as a campaign-style evaluation lab at CLEF.  ... 
doi:10.1145/3239570 dblp:journals/jdiq/HopfgartnerHMEB18 fatcat:6pe7jqqkh5hi3d3wmdly4gb5vi

WBI at CLEF eHealth 2018 Task 1: Language-independent ICD-10 Coding using Multi-lingual Embeddings and Recurrent Neural Networks

Jurica Seva, Mario Sänger, Ulf Leser
2018 Conference and Labs of the Evaluation Forum  
During evaluation our best model achieves an F-score of 0.34 for French, 0.45 for Hungarian and 0.77 for Italian.  ...  Our contribution focus on the setup and evaluation of a baseline language-independent neural architecture for ICD-10 classification as well as a simple, heuristic multi-language word embedding space.  ...  ICD-10 Classification The ICD-10 coding task has already been carried out in the 2016 [26] and 2017 [25] edition of the eHealth lab.  ... 
dblp:conf/clef/SevaSL18 fatcat:vjrfo3ugjnaqll5ztivpgd3phi

Annotation of Clinical Narratives in Bulgarian language

Ivaylo Radev, Kiril Simov, Galia Angelova, Svetla Boytcheva
2017 Proceedings of the Biomedical NLP Workshop  
The annotated corpus will be used as a Gold standard for information extraction evaluation of test corpus of 6,200 discharge letters.  ...  In this paper we describe annotation process of clinical texts with morphosyntactic and semantic information.  ...  Acknowledgments The research presented here is partially supported by the grant SpecialIZed Data MIning MethoDs Based on Semantic Attributes (IZIDA), funded by the Bulgarian National Science Fund in 2017  ... 
doi:10.26615/978-954-452-044-1_011 dblp:conf/ranlp/RadevSAB17 fatcat:bqbgov72ijch7cxt7gpae4kxpe

News Recommender System: A review of recent progress, challenges, and opportunities [article]

Shaina Raza, Chen Ding
2021 arXiv   pre-print
In the first part, we present an overview of the conventional recommendation solutions, datasets, evaluation criteria beyond accuracy and recommendation platforms being used in NRS.  ...  In order to help users to find the right and relevant content, news recommender systems (NRS) are developed to relieve the information overload problem and suggest news items that users might be interested  ...  Acknowledgment: This work is partially sponsored by Natural Science and Engineering Research Council of Canada (grant 2020-04760).  ... 
arXiv:2009.04964v4 fatcat:s7jl63nwm5e55myezsxpzquuje

2dSearch: A Visual Approach to Search Strategy Formulation

Tony Russell-Rose, Phil Gooch
2018 Biennial Conference on Design of Experimental Search & Information Retrieval Systems  
Knowledge workers (such as healthcare information professionals, patent agents and media monitoring professionals) need to create and execute search strategies that are accurate, repeatable and transparent  ...  Automated search term suggestions are provided using a combination of knowledge-based and statistical natural language processing techniques.  ...  ACKNOWLEDGMENTS Part of this work was funded by Technology Strategy Board grant 131641 "A visual framework for search query formulation" and Innovate UK grant 102975 "Intelligent search assistance".  ... 
dblp:conf/desires/Russell-RoseG18 fatcat:sumk6mf7ybahdbov2cpegtw25i

Hierarchical Deep Multi-modal Network for Medical Visual Question Answering [article]

Deepak Gupta, Swati Suman, Asif Ekbal
2020 arXiv   pre-print
We evaluate the performance of our proposed model on two benchmark datasets, viz. RAD and CLEF18.  ...  We also conduct a detailed quantitative and qualitative analysis of the obtained results and discover potential causes of errors and their solutions.  ...  We follow the evaluation setup discussed in ImageCLEF2018 VQA-Med 2018 challenge overview paper to evaluate the performance of the system.  ... 
arXiv:2009.12770v1 fatcat:d2dmtduat5b3bm4ujgyryh474y

Report on the Information Retrieval Festival (IRFest2017)

Leif Azzopardi, Craig Macdonald, Iadh Ounis, Martin Halvey
2017 SIGIR Forum  
The Information Retrieval Festival took place in April 2017 in Glasgow.  ...  The workshop was followed by a "Tour de Scotland" where delegates were taken from Glasgow to Aberdeen for the European Conference in Information Retrieval (ECIR 2017).  ...  Acknowledgements We would like to thank: the Scottish Informatics and Computing Science Alliance (SICSA); the University of Glasgow's 60th Anniversary Computing Science committee; and ACM SI-GIR executive  ... 
doi:10.1145/3130332.3130336 fatcat:3ejhaputzve5dileftpldba4uu

Question Answering in the Biomedical Domain

Vincent Nguyen
2019 Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop  
We find that there are some challenges in patient question answering such as limited annotated data, lexical gap and quality of answer spans.  ...  We aim to address some of these gaps by extending and developing upon the literature to design a question answering system that can decide on the most appropriate answers for patients attempting to self-diagnose  ...  NCBI at the 2014 BioASQ challenge task: Largescale biomedical semantic indexing and question answering. In Conference and Labs of the Evaluation Forum, Sheffield, UK.  ... 
doi:10.18653/v1/p19-2008 dblp:conf/acl/Nguyen19 fatcat:23yy6eohofh7tepzp4iixore2e
« Previous Showing results 1 — 15 out of 137 results