72,938 Hits in 6.8 sec

Retrieval system evaluation using recall and precision: problems and answers

V. V. Raghavan, P. Bollmann, G. S. Jung
1989 SIGIR Forum  
In particular, a recall-precision graph is often used as a combined evaluation measure of retrieval systems. Such a graph, given an arbitrary recall point, tells us the corresponding precision value.  ...  CONCLUSIONS Two interesting problems that arise, when using recall and precision as measures of retrieval system performance, are due to the weak ordering of output and the need for handling multiple queries  ... 
doi:10.1145/75335.75342 fatcat:ggunvvurdvcs3m5pqjeslm463m

A knowledge based method for the medical question answering problem

Rafael M. Terol, Patricio Martínez-Barco, Manuel Palomar
2007 Computers in Biology and Medicine  
The knowledge of the system is acquired through the use of two different resources: Unified Medical Language System (UMLS) to handle the medical terminology and WordNet to manage the open-domain terminology  ...  In this paper, a restricted domain Question Answering (QA) system is described.  ...  The fact that our QA system is only able to answer questions in this question taxonomy produces on one hand a lower recall but on the other hand a higher precision with the aim that our system will be  ... 
doi:10.1016/j.compbiomed.2007.01.013 pmid:17374369 fatcat:ze2r5lfqyvg2jdkxgy2ybi45ty

Adaptation of language model of information retrieval for empty answers problem in databases

Abdelhamid Chellal, Karima Amrouche
2015 2015 12th International Symposium on Programming and Systems (ISPS)  
Thereby, the user can be confronted to the problem of empty answers in the case of too selective query.  ...  deal with empty answers.  ...  An other work [13] suggests the use of the basic probabilistic model to rank answers for handling over-abundant answer problem.  ... 
doi:10.1109/isps.2015.7244977 fatcat:azt5ldcfhfexnn2cgbi2zz5nf4

Design of a Higher Education Question and Answer System Based on Multimodal Adversarial Networks

Xinyi Fu, Ning Cao
2022 Mathematical Problems in Engineering  
Most of the traditional quiz systems based on traditional retrieval techniques have problems such as insufficient semantic portrayal of text, inability to extract semantic features in context, and poor  ...  adversarial network-based question and answer system for higher education.  ...  retrieval, and answer extraction into a more complex question and answer system.  ... 
doi:10.1155/2022/7453653 fatcat:db2ers35lbcfnous3hwsbofiuy

Design and Implementation of a Medical Question and Answer System Based on Deep Learning

Yun Hu, Guokai Han, Xintang Liu, Hui Li, Libao Xing, Yong Gu, Zuojian Zhou, Haining Li, Lianhui Li
2022 Mathematical Problems in Engineering  
We took a retrieval-based approach, using crawler technology that has been manually reviewed to build the Q&A database, and the Seq2Seq algorithm and the TF-IDF model to build the answer generation model  ...  The medical question and answer system developed enable effective Q&A and relevant medical advice to be given.  ...  Common evaluation metrics for text classification tasks include accuracy, precision, recall, and F1-score to name a few. (1) Accuracy.  ... 
doi:10.1155/2022/4600404 fatcat:id57vgvj25axvf5aqm2i6yi54q

Is this Change the Answer to that Problem? Correlating Descriptions of Bug and Code Changes for Evaluating Patch Correctness [article]

Haoye Tian, Xunzhu Tang, Andrew Habib, Shangwen Wang, Kui Liu, Xin Xia, Jacques Klein, Tegawendé F. Bissyandé
2022 arXiv   pre-print
Concretely, we turn the patch correctness assessment into a Question Answering problem.  ...  In this work, we propose a novel perspective to the problem of patch correctness assessment: a correct patch implements changes that "answer" to a problem posed by buggy behaviour.  ...  Therefore, we use the two most common metrics, AUC and F1 score (harmonic mean between precision and recall for identifying correct patches), to evaluate the overall performance of our approach [16]  ... 
arXiv:2208.04125v1 fatcat:fa52dvtxlfgqtju2c6so2dzer4

Morphological Resources for Precise Information Retrieval [chapter]

Anne-Laure Ligozat, Brigitte Grau, Delphine Tribout
2012 Lecture Notes in Computer Science  
Question answering (QA) systems aim at providing a precise answer to a given user question. Their major difficulty lies in the lexical gap problem between question and answering passages.  ...  Then, we evaluate the results of a particular QA system, according to the morphological knowledge used.  ...  Introduction Question answering (QA) systems aim at providing a precise answer to a given user question.  ... 
doi:10.1007/978-3-642-32790-2_84 fatcat:3dh37jtkhrhnfn5vxmzi7knmly

Rank-biased precision for measurement of retrieval effectiveness

Alistair Moffat, Justin Zobel
2008 ACM Transactions on Information Systems  
Average precision is derived from recall, and suffers from the same problem. In addition, average precision lacks key stability properties that are needed for robust experiments.  ...  ACM Reference Format: Moffat, A. and Zobel, J. 2008. Rank-biased precision for measurement of retrieval effectiveness.  ...  ACKNOWLEDGMENTS Jamie Callan, Bruce Croft, Mark Sanderson, Ellen Voorhees, and William Webber provided helpful assistance.  ... 
doi:10.1145/1416950.1416952 fatcat:qpe7245dgfelvn5hwnjrjyuiuq

Improving Precision in Information Retrieval for Swedish using Stemming

Johan Carlberger, Hercules Dalianis, Martin Duneld, Ola Knutsson
2001 Nordic Conference of Computational Linguistics  
We will in this paper present an evaluation 1 of how much stemming improves precision in information retrieval for Swedish texts.  ...  Our final results were that stemming improved both precision and recall with 15 respectively 18 percent for Swedish texts having an average length of 181 words.  ...  allowing us to use their search engine in our experiments.  ... 
dblp:conf/nodalida/CarlbergerDDK01 fatcat:bnyzbxnsk5dstlwxmeuco7rgfa

User performance versus precision measures for simple search tasks

Andrew Turpin, Falk Scholer
2006 Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval - SIGIR '06  
Two of the studies used an instance recall task, and a third used a question answering task, so perhaps it is unsurprising that the precision based measures of IR system effectiveness on one-shot query  ...  Our results show that there is no significant relationship between system effectiveness measured by MAP and the precision-based task.  ...  However, mean average precision, while including a recall component, evaluates systems predom-inantly using precision [6, 19] . Similar observations hold true for metrics such as P@10.  ... 
doi:10.1145/1148170.1148176 dblp:conf/sigir/TurpinS06 fatcat:gbkbtfadhzabhcsfkszsxctryq

Extending average precision to graded relevance judgments

Stephen E. Robertson, Evangelos Kanoulas, Emine Yilmaz
2010 Proceeding of the 33rd international ACM SIGIR conference on Research and development in information retrieval - SIGIR '10  
Evaluation metrics play a critical role both in the context of comparative evaluation of the performance of retrieval systems and in the context of learning-to-rank (LTR) as objective functions to be optimized  ...  a graded precision-recall curve and it can be justified in terms of a simple but moderately plausible user model.  ...  [2] , for any query, we choose those systems that retrieved at least 5 relevant and 5 highly relevant documents to have a sufficient number of points on the precision-recall curves.  ... 
doi:10.1145/1835449.1835550 dblp:conf/sigir/RobertsonKY10 fatcat:yc3hf7j3ubdrpdlivzlmnmisyy

BIT.UA@TREC 2020 Precision Medicine Track

Tiago Almeida, Sérgio Matos
2020 Text Retrieval Conference  
To further explore and assess the effectiveness of deep learning methods in the PM retrieval task, we reformulate this relevance problem of evidence finding as a question-answering problem, where a query  ...  More precisely, we adopted a two-stage retrieval pipeline, where we first reduce the searching space using BM25 with gene name expansion and then apply a lightweight neural IR model, with only 620 trainable  ...  for Science and Technology, in the context of the project UIDB/00127/2020.  ... 
dblp:conf/trec/AlmeidaM20a fatcat:pwwt65dr3zf4nnvvdlrs2zn3fu

Beyond Precision: A Study on Recall of Initial Retrieval with Neural Representations [article]

Yan Xiao, Jiafeng Guo, Yixing Fan, Yanyan Lan, Jun Xu, Xueqi Cheng
2018 arXiv   pre-print
Our experiments show that both hybrid index and search schemes can improve the recall of the initial retrieval stage with small overhead.  ...  Therefore, in this paper, we study the problem how to employ neural representations to improve the recall of relevant documents in the initial retrieval stage.  ...  Meanwhile, they evaluated by the precision while not by the recall. To address the vocabulary mismatch problem, Boytsov et al.  ... 
arXiv:1806.10869v2 fatcat:f7ggl2nnszchzdhqmkupfc63y4

High Precision Latent Semantic Evaluation for Descriptive Answer Assessment

Amarjeet Kaur, M. Sasi Kumar
2018 Journal of Computer Science  
This paper proposes an approach to evaluate student's descriptive answers, using comparison-based approach in which student's answer is compared with the standard answer.  ...  With this as background, we investigated evaluation of students' descriptive answer using Latent Semantic Analysis (LSA).  ...  Latent Semantic Evaluation for Descriptive Answer Assessment".  ... 
doi:10.3844/jcssp.2018.1293.1302 fatcat:v7qbu27b6jbrrlrdf2zwxi7gje

HPS: High precision stemmer

Tomáš Brychcín, Miloslav Konopík
2015 Information Processing & Management  
We used corpora in the Czech, Slovak, Polish, Hungarian, Spanish and English languages.  ...  The second-stage algorithm uses a maximum entropy classifier. The stemming-specific features help the classifier decide when and how to stem a particular word.  ...  SGS-2013-029 Advanced computing and information systems, by the European Regional Development Fund (ERDF) and by project "NTIS -New Technologies for Information Society", European Centre of Excellence,  ... 
doi:10.1016/j.ipm.2014.08.006 fatcat:giqrh6znpfh6zduiq3p64g23ny
« Previous Showing results 1 — 15 out of 72,938 results