4,997 Hits in 5.9 sec

Building a reusable test collection for question answering

Jimmy Lin, Boris Katz
2006 Journal of the American Society for Information Science and Technology  
We have shown that the current evaluation resources have major shortcomings, and to remedy the situation, we have manually created a small, reusable question answering test collection for research purposes  ...  This article describes our methodology for building this test collection and discusses issues we encountered along the way regarding the notion of "answer correctness". † entities such as dates, locations  ...  Acknowledgements We would like to recognize Matthew Bilotti and Leah Oats for their tireless annotation effort. A portion of this work was the subject of Matthew's Master's thesis.  ... 
doi:10.1002/asi.20348 fatcat:qbky23za6nbmdipcdwppda3jta

Overview of the PAN/CLEF 2015 Evaluation Lab [chapter]

Efstathios Stamatatos, Martin Potthast, Francisco Rangel, Paolo Rosso, Benno Stein
2015 Lecture Notes in Computer Science  
In plagiarism detection, community-driven corpus construction is introduced as a new way of developing evaluation resources with diversity.  ...  A new corpus was built for this challenging, yet realistic, task covering four languages.  ...  Acknowledgements We thank the organizing committees of PAN's shared tasks Fabio Celli, Walter Daelemans, Ben Verhoeven, Patrick Juola, and Aurelio López-López.  ... 
doi:10.1007/978-3-319-24027-5_49 fatcat:fcpf2p7nujet5ez4zswoiscatq

Quantum, a French/English Cross-Language Question Answering System [chapter]

Luc Plamondon, George Foster
2004 Lecture Notes in Computer Science  
Our method relies on a statistical translation engine to translate keywords, and a set of manually written rules for analyzing French questions.  ...  We describe a method for modifying a monolingual English question answering system to allow it to accept French questions.  ...  Acknowledgements This project was financially supported by the Bell University Laboratories, the Natural Science and Engineering Council of Canada and the National Research Council of Canada.  ... 
doi:10.1007/978-3-540-30222-3_53 fatcat:2rifhygq5rdezlxwq2pbr2o6ae

Multistage BiCross encoder for multilingual access to COVID-19 health information

Iknoor Singh, Carolina Scarton, Kalina Bontcheva, Balaraman Ravindran
2021 PLoS ONE  
The Coronavirus (COVID-19) pandemic has led to a rapidly growing 'infodemic' of health information online.  ...  This has motivated the need for accurate semantic search and retrieval of reliable COVID-19 information across millions of documents, in multiple languages.  ...  For each language, the corpus consists of general health-related articles collected from different websites, out of which the majority of articles come from the Medical Information System (MEDISYS) https  ... 
doi:10.1371/journal.pone.0256874 pmid:34492073 fatcat:wl5ozj47s5aajpilg7jobwxhka

What Happened in CLEF2009: Introduction to the Working Notes

Carol Peters
2009 Conference and Labs of the Evaluation Forum  
This is done through the organisation of annual evaluation campaigns in which a series of tracks designed to test different aspects of mono-and cross-language information retrieval (IR) are offered.  ...  The intention is to encourage experimentation with all kinds of multilingual information access -from the development of systems for monolingual retrieval operating on many languages to the implementation  ...  The track focused on the task of prior art search. A large test collection for evaluation purposes was created by exploiting patent citations.  ... 
dblp:conf/clef/Peters09b fatcat:p5tpt2abovetffp3tjip2n3n6a

CLEF, CLEF 2010, and PROMISEs: Perspectives for the Cross-Language Evaluation Forum

Nicola Ferro
2010 NTCIR Conference on Evaluation of Information Access Technologies  
This paper will provide a brief summary of the most significant results achieved by CLEF in the past ten years, it will describe the new format and organization for CLEF which is being experimented for  ...  After ten years of increasingly successful evaluation campaigns, the Cross-Language Evaluation Forum (CLEF) has come to an appropriate moment to assess what has been achieved in this decade and also to  ...  The track focused on the task of prior art search. A large test collection for evaluation purposes was created by exploiting patent citations.  ... 
dblp:conf/ntcir/Ferro10 fatcat:xn5ttzldifbevbvvr3jszzelhi

Document Expansion for Cross-Lingual Passage Retrieval

Eneko Agirre, Olatz Ansa, Xabier Arregi, Maddalen Lopez de Lacalle, Arantxa Otegi, Xabier Saralegi
2010 Conference and Labs of the Evaluation Forum  
for Basque to English retrieval when faced with the lack of parallel corpora for Basque in this domain, and (3) to check the contribution of semantic relatedness based on WordNet to expand the passages  ...  to related words.  ...  Arantxa Otegi's work is funded by a PhD grant from the Basque Government.  ... 
dblp:conf/clef/AgirreAALOS10 fatcat:cqfzuyswrzak5nrvx7ryjtuquy

Resolving ambiguity for cross-language retrieval

Lisa Ballesteros, W. Bruce Croft
1998 Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval - SIGIR '98  
One of the main hurdles to improved CLIR effectiveness is resolving ambiguity associated with translation. Availability of resources is also a problem.  ...  Finally, we compare the co-occurrence method with parallel corpus and machine translation techniques and show that good retrieval effectiveness can be achieved without complex resources.  ...  Acknowledgments This material is based on work supported by the National Science Foundation, Library of Congress, and Department of Commerce under cooperative agreement number EEC-9209623.  ... 
doi:10.1145/290941.290958 dblp:conf/sigir/BallesterosC98 fatcat:6o7msk3edvdl7mfvfghzwprsia

Arabic Question Answering Systems: Gap Analysis

Mariam M. Biltawi, Sara Tedmori, Arafat Awajan
2021 IEEE Access  
For example, if two candidate answers with two different relations exist, then the passage having a relation with higher priority is selected as an answer; if both passages have the same relation or relations  ...  ., "How many ways can you study for a test?"), and evaluation questions (e.g., "Do you agree with …?" and "What is your opinion on …?").  ... 
doi:10.1109/access.2021.3074950 fatcat:zsjkex2ylfa5hgg3qjedhqsce4

SINAI at CLEF eHealth 2018 Task 3. Using cTAKES to Remove Noise from Expanding Queries with Google

Manuel Carlos Díaz-Galiano, Pilar López-Úbeda, María Teresa Martín-Valdivia, Luis Alfonso Ureña López
2018 Conference and Labs of the Evaluation Forum  
We also significantly reduces the response time of the Information Retrieval System (IRS) by 90% compared to previous years.  ...  The main objective of the task is to provide relevant information to people seeking health advice on the web.  ...  Acknowledgments This work has been partially supported by a grant from Fondo Europeo de Desarrollo Regional (FEDER) and REDES project (TIN2015-65136-C2-1-R) from the Spanish Government.  ... 
dblp:conf/clef/Diaz-GalianoLML18 fatcat:kvxtilnym5hjrlobjpwmm6auyi

TREC-COVID: Rationale and Structure of an Information Retrieval Shared Task for COVID-19

Kirk Roberts, Tasmeer Alam, Steven Bedrick, Dina Demner-Fushman, Kyle Lo, Ian Soboroff, Ellen Voorhees, Lucy Lu Wang, William R Hersh
2020 JAMIA Journal of the American Medical Informatics Association  
, relevance criteria, evaluation metrics, iteration process, projected timeline, and the implications of data use as a post-task test collection.  ...  This article describes how all these were addressed for the particular requirements of developing IR systems under a pandemic situation.  ...  for funding support.  ... 
doi:10.1093/jamia/ocaa091 pmid:32365190 pmcid:PMC7239098 fatcat:rhqp7cvirrgvpfcyz7zji4z67u

Overview of BioASQ 2021: The ninth BioASQ challenge on Large-Scale Biomedical Semantic Indexing and Question Answering [article]

Anastasios Nentidis, Georgios Katsimpras, Eirini Vandorou, Anastasia Krithara, Luis Gasco, Martin Krallinger, Georgios Paliouras
2021 arXiv   pre-print
The evaluation results, similarly to previous years, show a performance gain against the baselines which indicates the continuous improvement of the state-of-the-art in this field.  ...  This paper presents an overview of the ninth edition of the BioASQ challenge in the context of the Conference and Labs of the Evaluation Forum (CLEF) 2021.  ...  Finally, for the patents subtrack (MESINESP-P), the aim was to explore and evaluate indexing strategies of medical patents written in Spanish providing only a very small manually annotated patent collection  ... 
arXiv:2106.14885v1 fatcat:uix2mc6jh5hg3jegwzfhwblzc4

Biomedical Question Answering: A Survey of Approaches and Challenges [article]

Qiao Jin, Zheng Yuan, Guangzhi Xiong, Qianlan Yu, Huaiyuan Ying, Chuanqi Tan, Mosha Chen, Songfang Huang, Xiaozhong Liu, Sheng Yu
2021 arXiv   pre-print
There have been tremendous developments of BQA in the past two decades, which we classify into 5 distinctive approaches: classic, information retrieval, machine reading comprehension, knowledge base and  ...  In this survey, we introduce available datasets and representative methods of each BQA approach in detail.  ...  In general, QA itself is a challenging benchmark Natural Language Processing (NLP) task for evaluating the abilities of intelligent systems to understand a question, retrieve and utilize relevant materials  ... 
arXiv:2102.05281v2 fatcat:gngt6kffszbzbc3donik55dd64

Artificial Intelligence in Action: Addressing the COVID-19 Pandemic with Natural Language Processing

Qingyu Chen, Robert Leaman, Alexis Allot, Ling Luo, Chih-Hsuan Wei, Shankai Yan, Zhiyong Lu
2021 Annual Review of Biomedical Data Science  
The COVID-19 (coronavirus disease 2019) pandemic has had a significant impact on society, both because of the serious health effects of COVID-19 and because of public health measures implemented to slow  ...  Expected final online publication date for the Annual Review of Biomedical Data Science, Volume 4 is July 2021. Please see for revised estimates.  ...  ACKNOWLEDGMENTS This research is supported by the NIH Intramural Research Program, National Library of Medicine.  ... 
doi:10.1146/annurev-biodatasci-021821-061045 pmid:34465169 fatcat:gxtcwu5ih5gx3la6ea4674pnmu

A logical representation of Arabic questions toward automatic passage extraction from the Web

Wided Bakari, Patrice Bellot, Mahmoud Neji
2017 International Journal of Speech Technology  
And then, the results of passages of texts that are automatically generated achieved an 87% score for accuracy and a 98% score for c@1.  ...  Next, it is essential to retrieve a passage from the web that can serve as an appropriate answer.  ...  = (1) Second, we tested our proposed method for Answer passage retrieval with a set of collected questions, which consists of 115 factual questions.  ... 
doi:10.1007/s10772-017-9411-7 fatcat:u7wsti7bgrb43jt6auaawy4ziy
« Previous Showing results 1 — 15 out of 4,997 results