A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL.
The file type is application/pdf.
Filters
Cumulated gain-based evaluation of IR techniques
2002
ACM Transactions on Information Systems
Preserved Fulltext
The third one computes the relative-to-the-ideal performance of IR techniques, based on the cumulative gain they are able to yield. ...
The graphs based on the measures also provide insight into the performance IR techniques and allow interpretation, e.g., from the user point of view. ...
ACKNOWLEDGEMENTS We thank the FIRE group at University of Tampere for helpful comments, and the IR Lab for programming. ...
doi:10.1145/582415.582418
fatcat:jheqo3rghbenxd75q42ztuuqli
Binary and graded relevance in IR evaluations—Comparison of the effects on ranking of IR systems
2005
Information Processing & Management
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2006; you can also visit the original URL.
The file type is application/pdf.
In this study the rankings of IR systems based on binary and graded relevance in TREC 7 and 8 data are compared. ...
Binary precision, and cumulated gain, discounted cumulated gain and normalised discounted cumulated gain are the measures compared. ...
Acknowledgment This study was funded by Academy of Finland under the grant number 52894. ...
doi:10.1016/j.ipm.2005.01.004
fatcat:brackldjarftfiijilvgnrbsti
Cumulated Relative Position: A Metric for Ranking Evaluation
[chapter]
2012
Lecture Notes in Computer Science
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL.
The file type is application/pdf.
IR research offers a strong evaluation methodology and a range of evaluation metrics, such as MAP and (n)DCG. In this paper, we propose a new metric for ranking evaluation, the CRP. ...
By cumulating these relative rankings we indicate, at each ranked position, the net effect of document displacements, the CRP. ...
Acknowledgements The work reported in this paper has been supported by the PROMISE network of excellence (contract n. 258191) project as a part of the 7th Framework Program of the European commission ( ...
doi:10.1007/978-3-642-33247-0_13
fatcat:cmt2sepjavgsdb7somd2sgfyly
Intuition-supporting visualization of user's performance based on explicit negative higher-order relevance
2008
Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval - SIGIR '08
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL.
The file type is application/pdf.
Modeling the beyond-topical aspects of relevance are currently gaining popularity in IR evaluation. ...
In this paper, we focus on the concept of negative higher-order relevance (NHOR) made explicit via negative gain values in IR evaluation. ...
If only the conventional nonnegative gain values are allowed, the normalized values are in the range [0, 1] and they express the share of ideal performance cumulated [4] by the technique evaluated. ...
doi:10.1145/1390334.1390448
dblp:conf/sigir/KeskustaloJPK08
fatcat:atra6sikd5ftbmrmyho22xvrqu
Performance Assessment of AMRR and ADCG Metrics in MLIR and IR Systems
2011
International Journal of Computer Applications
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL.
The file type is application/pdf.
The experimental results are evaluated to analyze and compare the performance of proposed MLIR system metrics, Average Mean Reciprocal Rank (AMRR) and Average Discounted Cumulative Gain (ADCG), in Information ...
Experimental results show that the performance of AMRR, ADCG in MLIR system has been improved 81.67%, 43.93% over IR system respectively. ...
gain after that evaluated discounted cumulative gain. ...
doi:10.5120/2919-3851
fatcat:praeniu7r5a35ptvfz4sqe3s6q
Test-framework: performance profiling and testing web search engine on non factoid queries
2019
Indonesian Journal of Electrical Engineering and Computer Science
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
of arts approaches in terms of set based measures like Precision, Recall and F measure and rank based measures like Mean Average Precision and Cumulative Gain. ...
This stands difficult to evaluate the retrieval method in order to schedule or optimize the Recommendation and prediction technique of the IR method to the Real time queries. ...
engine evaluation against mean average precision
1380
Figure 6 . 6 Performance analysis of search engine evaluation against the cumulative gain The
Indonesian J Elec Eng & Comp SciTest-retrieval ...
doi:10.11591/ijeecs.v14.i3.pp1373-1381
fatcat:42kxp6frazfmnltvir5nq36awu
A Framework for the Evaluation of Adaptive IR Systems through Implicit Recommendation
[chapter]
2011
Lecture Notes in Computer Science
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL.
The file type is application/pdf.
A Review of evaluation techniques for personalised IR systems was conducted and the results of the analysed survey are presented. ...
This paper presents a web-based adaptive framework for evaluating personalised information retrieval systems. ...
Acknowledgements This research is based upon works supported by Science Foundation Ireland (Grant Number: 07/CE/I1142) as part of the Centre for Next Generation Localization (www.cngl.ie). ...
doi:10.1007/978-3-642-22688-5_32
fatcat:i5r67jst5fek7op6xm67khlqlu
Visual Comparison of Ranked Result Cumulated Gains
[article]
2011
International EuroVis Workshop on Visual Analytics
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL.
The file type is application/pdf.
the cumulative gain up to a given ranked position and taking into account multiple relevance levels. ...
Ranking is fundamental in Information Retrieval (IR) and several measures have been developed over the years to assess the quality of a ranked result list, such as those based on the idea of computing ...
Finally, the idea itself of exploring and applying VA techniques to the experimental evaluation in the IR field is quite innovative since it has never been attempted before and, due to the complexity of ...
doi:10.2312/pe/eurovast/eurova11/021-024
dblp:conf/eurova-ws/FerroSSTV11
fatcat:k3yuvzgfeze37fb2fyklikcfgu
Evaluating information retrieval system performance based on user preference
2009
Journal of Intelligent Information Systems
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL.
The file type is application/pdf.
The main objective of this paper is to propose a framework for system evaluation based on user preference of documents. ...
This calls for choosing the proper methods to evaluate the system performance. ...
We find that most evaluation methods are based on cumulated gain. ...
doi:10.1007/s10844-009-0096-5
fatcat:i5qrpg3acnaq3ln3fspl7svsvq
eXtended cumulated gain measures for the evaluation of content-oriented XML retrieval
2006
ACM Transactions on Information Systems
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2011; you can also visit the original URL.
The file type is application/pdf.
We propose and evaluate a family of metrics, named the eXtended Cumulated Gain (XCG) metrics, for the evaluation of content-oriented XML retrieval approaches. ...
We evaluate the proposed XCG metrics, based on the INEX test collection, both with respect to their fidelity and reliability. ...
ACKNOWLEDGEMENTS We are thankful for the help of Paul Ogilvie and Saadia Malik on some of the scripts for the R statistical package. ...
doi:10.1145/1185877.1185883
fatcat:lj6dxirjtncn5fpu6xtuamk3ru
Visual Analytics for Information Retrieval Evaluation (VAIRË 2015)
[chapter]
2015
Lecture Notes in Computer Science
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL.
The file type is application/pdf.
The tutorial introduced basic and intermediate concepts about lab-based evaluation of information retrieval systems, its pitfalls, and shortcomings and it complemented them with a recent and innovative ...
Retrieval (IR) system behaviour. ...
Some examples of well-known measures, such as Average Precision (AP), Normalized Discounted Cumulated Gain (nDCG), Rank-Biased Precision (RBP) and so on, have been discussed [4] . ...
doi:10.1007/978-3-319-16354-3_91
fatcat:aqa3qqai5va55nirrj5bwkkfe4
Beyond classical measures
2007
Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval - SIGIR '07
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
Previous studies have concluded that improvements in IR systems based on increase in IR effectiveness measures do not reflect on improvement in users' performance. ...
Thus, by the end of this research, we hope to develop a valid and reliable metrics for IIR evaluation. ...
Thus, we set up a second experiment to investigate how users' satisfaction correlate with some IR effectiveness measures such as precision and the suite of Cumulative Gain measures (CG, DCG, NDCG) in simple ...
doi:10.1145/1277741.1277984
dblp:conf/sigir/Al-Maskari07
fatcat:mfxegxtsevafvgzahrchjica4e
Improving Ranking Evaluation Employing Visual Analytics
[chapter]
2013
Lecture Notes in Computer Science
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL.
The file type is application/pdf.
This paper provides an analytical model for examining rankings produced by IR systems, based on the discounted cumulative gain family of metrics, and visualization for performing failure and "what-if" ...
The work reported in this paper has been supported by the PROMISE network of excellence (contract n. 258191) project as a part of the 7th Framework Program of the European commission (FP7/2007-2013). ...
Related Work The graded-relevance metrics considered in this paper are based on cumulative gain [5] ; the Discounted Cumulated Gain (DCG) measures are based on the idea that documents are divided in multiple ...
doi:10.1007/978-3-642-40802-1_4
fatcat:63vqdlia2rbi7cyjyugk5jj2my
Performance evaluation of non-Gaussian channel estimation techniques on ultrawideband impulse radio channels
2007
International Journal of Communication Systems
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2004; you can also visit the original URL.
The file type is application/pdf.
Modern UWB-IR technology is based on transmission of ultra-short pulses with very broad but low-amplitude power spectral densities (PSDs) [1, 2] . ...
The purpose of the work is to demonstrate that the statistical characteristics of the UWB-IR environment are strongly non-Gaussian and that as a result, non-Gaussian signal processing techniques are both ...
In this section, we evaluate the performance of the three training-sequence-based channel estimation algorithms. ...
doi:10.1002/dac.861
fatcat:yhhpr4jr4beozmotcejlvnjvhm
IR Research: Systems, Interaction, Evaluation and Theories
[chapter]
2011
Lecture Notes in Computer Science
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL.
The file type is application/pdf.
We will discuss nested IR evaluation frameworks ranging from IR system centered evaluation to work-task based evaluation. ...
Such studies do not solve a technical problem, nor evaluate any particular technique, and may therefore be considered unpractical. ...
Paper Award *) for session-based IR evaluation; the IIiX 2010 Best Paper Award *) for a study on task-based information access; and the Tony Kent Strix Award 2008 in recognition of contributions to the ...
doi:10.1007/978-3-642-20161-5_1
fatcat:ndk75j2lrrad7l76awq6qqpb2m
« Previous
Showing results 1 — 15 out of 28,096 results