On evaluation of adaptive topic tracking systems

Tamer Elsayed, Douglas W. Oard
2005 Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval - SIGIR '05  
Summative evaluation methods for supervised adaptive topic tracking systems convolve the effect of system decisions on present utility with the effect on future utility. This paper describes a new formative evaluation approach that focuses on future utility for use in the design stage of adaptive systems. Topic model quality is assessed at a predefined set of points using a fixed document set to enhance comparability. Experiments using a vector-space topic tracking system illustrate the utility of this approach to formative evaluation.
doi:10.1145/1076034.1076146 dblp:conf/sigir/ElsayedO05 fatcat:mb4np45hnjdjllneo546laneh4