Performance evaluation in content-based image retrieval: overview and proposals

Henning Müller, Wolfgang Müller, David McG. Squire, Stéphane Marchand-Maillet, Thierry Pun
2001 Pattern Recognition Letters  
Evaluation of retrieval performance is a crucial problem in content-based image retrieval CBIR. Many di erent methods for measuring the performance of a system have been created and used by researchers. This article discusses the advantages and shortcomings of the performance measures currently used. Problems such a s a commonimage database for performance comparisons and a means of getting relevance judgments or ground truth for queries are explained. The relationship between CBIR and
more » ... on retrieval IR is made clear, since IR researchers have decades of experience with the evaluation problem. Many of their solutions can be used for CBIR, despite the di erences between the elds. Several methods used in text retrieval are explained. Proposals for performance measures and means of developing a standard test suite for CBIR, similar to that used in IR at the annual Text REtrieval Conference TREC, are presented.
doi:10.1016/s0167-8655(00)00118-5 fatcat:q3zsturvkndvppag3adln2pmqa