42,426 Hits in 8.6 sec

Statistical Significance Testing in Information Retrieval

Julián Urbano, Harlley Lima, Alan Hanjalic
2019 Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval - SIGIR'19  
over the null hypotheses to compute actual Type I and Type II error rates under realistic conditions.  ...  Statistical significance testing is widely accepted as a means to assess how well a difference in effectiveness reflects an actual difference between systems, as opposed to random noise because of the  ...  Whichever tree, plant, or bird you're now part of, thank you, Mom.  ... 
doi:10.1145/3331184.3331259 dblp:conf/sigir/UrbanoLH19 fatcat:32kkxiuyyzf7xmsszi7agsdmfa


Sadanandam Manchala
2012 IOSR Journal of Engineering  
The efficiency of retrieval system is precise by comparing performance on a regular set of queries in Information Retrieval (IR) and MLIR systems.  ...  Our results show that previous experimental work on significance tests over-estimated the error of such tests.  ...  Apart from correctly determining significance or a lack thereof, the tests also produce type I and type II errors.  ... 
doi:10.9790/3021-0204794802 fatcat:furywk2wzzbdbhyyuqzzenq5hu

Improving Web Search Using Contextual Retrieval

Dilip K. Limbu, Andrew M. Connor, Russel Pears, Stephen G. MacDonell
2009 2009 Sixth International Conference on Information Technology: New Generations  
Contextual retrieval is a critical technique for today's search engines in terms of facilitating queries and returning relevant information.  ...  An empirical study has been undertaken to evaluate the system against a number of hypotheses.  ...  The empirical study comprised three phases OS-I, OS-II, and OS-III. The OS-I and OS-II phases were carried out with differing objectives.  ... 
doi:10.1109/itng.2009.133 dblp:conf/itng/LimbuCPM09 fatcat:gaumdzclp5b7topwknbmzxphga

Improving web search using contextual retrieval [article]

Dilip K. Limbu, Andy M. Connor, Russel Pears, Stephen G. MacDonell
2014 arXiv   pre-print
Contextual retrieval is a critical technique for today's search engines in terms of facilitating queries and returning relevant information.  ...  An empirical study has been undertaken to evaluate the system against a number of hypotheses.  ...  The empirical study comprised three phases OS-I, OS-II, and OS-III. The OS-I and OS-II phases were carried out with differing objectives.  ... 
arXiv:1407.6101v1 fatcat:sdd7aut2zbg23hxwwh6dwxexhe

Using score distributions to compare statistical significance tests for information retrieval evaluation

Javier Parapar, David E. Losada, Manuel A. Presedo‐Quindimil, Alvaro Barreiro
2019 Journal of the Association for Information Science and Technology  
The sign test and Wilcoxon signed test also have a good behavior in terms of type I errors. The bootstrap test shows few type I errors, but it has less power than the other methods tested.  ...  This new method for studying the power of significance tests in Information Retrieval evaluation is formal and innovative.  ...  Acknowledgements This work has received financial support from the i) "Ministerio de Economía y Competitividad" of the Government of Spain and FEDER Funds under the research project TIN2015-64282-R, ii  ... 
doi:10.1002/asi.24203 fatcat:jgohta6wmvfhbm3owh4zdoq42u

Information About Information: Public Investments in Information Retrieval Research

Albert N. Link, Brent R. Rowe, Dallas W. Wood
2011 Journal of the Knowledge Economy  
Information retrieval (IR) is the science and practice of matching information seekers with the information being sought.  ...  Research on IR focuses on improving the effectiveness and efficiency of retrieval techniques and evaluating competing retrieval mechanisms.  ...  Information about Information: Public Investments in Information Retrieval Research I.  ... 
doi:10.1007/s13132-011-0046-7 fatcat:nozd4fabrrfqto3ekbxu4x4mtu

Dependencies: Formalising Semantic Catenae for Information Retrieval [article]

Christina Lioma
2017 arXiv   pre-print
These tools are principally expressed in nine distinct models that capture aspects of semantic dependence in highly interpretable and non-complex ways.  ...  The amalgamation of the body of work presented in this dissertation advances the complexity and granularity of semantic inferences that can be made automatically by machines.  ...  the theoretical novelty of Model II, the significance of its findings to information retrieval can be summarised as follows: On a conceptual level it points out a significant error in the estimation of  ... 
arXiv:1709.03742v1 fatcat:4fdrnsmwdnb4pe37b6ritmvnme

Assessing the Impact of OCR Errors in Information Retrieval [chapter]

Guilherme Torresan Bazzo, Gustavo Acauan Lorentz, Danny Suarez Vargas, Viviane P. Moreira
2020 Lecture Notes in Computer Science  
In this empirical study, we simulate OCR errors and investigate the impact that misspelled words have on retrieval accuracy.  ...  A significant amount of the textual content available on the Web is stored in PDF files.  ...  This work was partially supported by Petrobras, CNPq/Brazil, and by CAPES Finance Code 001.  ... 
doi:10.1007/978-3-030-45442-5_13 fatcat:uignn2cccfc7dlko4yhxppykae

Advancing Trace Recovery Evaluation - Applied Information Retrieval in a Software Engineering Context [article]

Markus Borg
2016 arXiv   pre-print
To tackle this issue, several researchers have proposed treating the capture and recovery of trace links as an Information Retrieval (IR) problem.  ...  Also, this thesis contributes to the body of empirical evidence of IR-based trace recovery in two experiments with industrial software artifacts.  ...  While these results were analyzed using statistical testing, the low number of subjects did not result in any statistically significant results.  ... 
arXiv:1602.07633v1 fatcat:a5df75muvzhkfjjwrllptusc4i

Retrieving leaf area index from SPOT4 satellite data

M. Aboelghar, S. Arafat, A. Saleh, S. Naeem, M. Shirbeny, A. Belal
2010 Egyptian Journal of Remote Sensing and Space Sciences  
The accuracy of the generated models ranged between 50% in the case of Sakha-104 and 82% in the case of Giza-178. LAI maps were produced from NDVI imageries based on the generated models.  ...  Statistical analyses were performed to confirm the assumptions of inversion modeling for plant variables and to get reliable models that fit the inversion relationship between LAI and NDVI.  ...  The standard errors are smaller and the weighted sum of squared residuals is: S ¼ X n i¼1 W ii r 2 i ; W ii ¼ 1 r 2 i ð4Þ While S is the weighted sum of squared residuals.  ... 
doi:10.1016/j.ejrs.2010.06.001 fatcat:xy2twnpjxzcb7nbnildpcup2tq

Methods for Evaluating Interactive Information Retrieval Systems with Users

Diane Kelly
2007 Foundations and Trends in Information Retrieval  
] book on information seeking and retrieval are great background reading for those interested in the evolution of IIR systems and evaluation.  ...  In addition to the sources from the IIR and IR literature, a number of sources related to experimental design and statistics were instrumental in the development of this paper: Babbie [13], Cohen [56],  ...  Acknowledgments I would like to thank Nick Belkin and Paul Kantor for their training and guidance; Justin Zobel, Barbara Wildemuth and Cassidy Sugimoto for their feedback and discussion about this paper  ... 
doi:10.1561/1500000012 fatcat:w2ek674zgfbhlnhorrklwmbuyy

Variations on language modeling for information retrieval

Wessel Kraaij
2005 SIGIR Forum  
Variations on Language Modeling for Information Retrieval W. Kraaij -Enschede: Neslia Paniculata. Thesis Enschede -With ref. With summary ISBN 90-75296-09-6  ...  The background of the debate is the trade-off between Type I and Type II errors.  ...  An ideal test would have low values for both type I and type II errors, but as usual there is a trade-off. A lower α level will decrease the power of the test.  ... 
doi:10.1145/1067268.1067291 fatcat:h23lp5aqfvfu5iecwnihfme244

Analysis of Statistical Question Classification for Fact-Based Questions

Donald Metzler, W. Bruce Croft
2005 Information retrieval (Boston)  
Question classification systems play an important role in question answering systems and can be used in a wide range of other domains.  ...  Finally, we analyze common causes of misclassification error and provide insight into ways they may be overcome.  ...  Acknowledgments This work was supported in part by the Center for Intelligent Information Retrieval, in part by NSF grant number DUE-0226144 and in part by Advanced Research and Development Activity under  ... 
doi:10.1007/s10791-005-6995-3 fatcat:3m62nqnvgvbgxcuz7laybsr6ie

Minimum Probability of Error Image Retrieval

N. Vasconcelos
2004 IEEE Transactions on Signal Processing  
Minimum probability of error (MPE) is adopted as the optimality criterion and retrieval formulated as a problem of statistical classification.  ...  The probability of retrieval error is lower-and upper-bounded by functions of the Bayes and density estimation errors, and the impact of the components of the retrieval architecture (namely, the feature  ...  In Section II, we formulate the retrieval problem as one of supervised learning and review relevant results from learning theory. Section III addresses the issue of MPE image representation.  ... 
doi:10.1109/tsp.2004.831125 fatcat:2bmgag7qjjgynft2c3o7cl6v6y

An analysis on document length retrieval trends in language modeling smoothing

David E. Losada, Leif Azzopardi
2007 Information retrieval (Boston)  
First, we theoretically analyze the Jelinek-Mercer, Dirichlet prior and twostage smoothing strategies and, then, conduct an empirical analysis.  ...  In this article, we perform an in-depth study of this behavior, characterized by the document length retrieval trends, of three popular smoothing methods across a number of factors, and its impact on the  ...  Mark Baillie and the anonymous reviewers for their useful comments and suggestions which have been incorporated into this article. David E.  ... 
doi:10.1007/s10791-007-9040-x fatcat:5ypi4mtdyzhytn4ncen4lzrfue
« Previous Showing results 1 — 15 out of 42,426 results