Filters








1 Hit in 0.1 sec

Assessing the Quality of Opinion Retrieval Systems

Giambattista Amati, Giuseppe Amode, Valerio Capozio, Giorgio Gambosi, Carlo Gaibisso
2010 2010 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology  
Objectives of the work Topical Opinion Retrieval (TOR) is evaluated by classical IR evaluation measures, i.e. Mean Average Precision (MAP) or Precision at 10 (P@10). The effectiveness of the topical-only retrieval (effectiveness of the baseline) boosts the TOR performance. How can we assess the opinion-only classification accuracy (or precision, etc.)? How can we split the contribution of the opinion component from retrieval? Methodological Framework We build artificial opinion-only classifiers
more » ... on-only classifiers from relevance and opinion data at different rates of opinion accuracy and precision. Then we study the effect on MAP of the TOR system with such classifiers We are able to assess the opinion-only component quality of a given TOR system by comparing it with such artificial TOR systems. Summary Objectives of the work Topical Opinion Retrieval (TOR) is evaluated by classical IR evaluation measures, i.e. Mean Average Precision (MAP) or Precision at 10 (P@10). The effectiveness of the topical-only retrieval (effectiveness of the baseline) boosts the TOR performance. How can we assess the opinion-only classification accuracy (or precision, etc.)? How can we split the contribution of the opinion component from retrieval? Methodological Framework We build artificial opinion-only classifiers from relevance and opinion data at different rates of opinion accuracy and precision. Then we study the effect on MAP of the TOR system with such classifiers We are able to assess the opinion-only component quality of a given TOR system by comparing it with such artificial TOR systems. Then we study the effect on MAP of the TOR system with such classifiers We are able to assess the opinion-only component quality of a given TOR system by comparing it with such artificial TOR systems.
doi:10.1109/wi-iat.2010.272 dblp:conf/iat/AmatiACGG10 fatcat:7wv4hj666fadraoopsk7deye4e