Lessons from BMIR-J2

Tsuyoshi Kitani, Tetsuya Sakai, Takenobu Tokunaga, Hiroshi Tsuruoka, Hidekazu Nakawatase, Teru Agata, Yasushi Ogawa, Tetsuya Ishikawa, Haruo Kimoto, Ikuo Keshi, Jun Toyoura, Toshikazu Fukushima (+2 others)
1998 Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval - SIGIR '98  
BMIR-JP is the lirat complete Japanese test collection available for use in evaluating information retrieval systems. It contains sixty queries and the IDS of 5080 newspaper articles in the fields of economics and engineering. The queries are classified into five categories, based on the functions the system is likely to use to interpret them correctly and retrieve relevant texts. This collection has two levels of relevance, topically relevant and partially relevant. Also discussed are design
more » ... sues such as collection types and size. This collection and the principles derived in designing it should be helpful in the future development of new test collections.
doi:10.1145/290941.291033 dblp:conf/sigir/KitaniOIKKTFMUSTTNA98 fatcat:n6lyhfar6vbtxhlqxehrdvw6zm