Investigating the exhaustivity dimension in content-oriented XML element retrieval evaluation

Paul Ogilvie, Mounia Lalmas
2006 Proceedings of the 15th ACM international conference on Information and knowledge management - CIKM '06  
INEX, the evaluation initiative for content-oriented XML retrieval, has since its establishment defined the relevance of an element according to two graded dimensions, exhaustivity and specificity. The former measures how exhaustively an XML element discusses the topic of request, whereas specificity measures how focused the element is on the topic of request. However, obtaining relevance assessments is a costly task. In XML retrieval this problem is exacerbated as the elements of the document
more » ... ust also be assessed with respect to the exhaustivity and specificity dimensions. A continuous discussion in INEX has been whether such a sophisticated definition of relevance, and in particular the exhaustivity dimension, was needed. This paper attempts to answer this question through extensive statistical tests to compare the conclusions about system performance that could be made under different assessment scenarios.
doi:10.1145/1183614.1183631 dblp:conf/cikm/OgilvieL06 fatcat:fj2ik7me3rdzpm74nzycrdwsha