Overview of the NTCIR-5 WEB Navigational Retrieval Subtask 2 (Navi-2)

Keizo Oyama, Masao Takaku, Haruko Ishikawa, Akiko Aizawa, Hayato Yamana
2005 NTCIR Conference on Evaluation of Information Access Technologies  
This paper describes an overview of the Navigational Retrieval Subtask 2 that was conducted from 2004 to 2005 as a subtask of the WEB Task at the Fifth NTCIR Workshop. In the Subtask, we attempted to assess the retrieval effectiveness of web search systems from a viewpoint of "Known Item Search" using a common data set, and built a re-usable test collection. 1.36TB web document data and 400 topics were distributed to the participants and, in turn, 35 run results were submitted by 4 participants
more » ... and 28 by the organizers. Relevance judgments were performed on the documents pooled from the run results, mainly in terms of representativeness of search target items given by the topics. Several kinds of evaluation measures were applied to the run results submitted by each participant. Simple analyses on system evaluation results and on test collection characteristics are given.
dblp:conf/ntcir/OyamaTIAY05 fatcat:ft3yc42u7zfxdeeopn7bi6cyla