Overview of the NTCIR-13 QA Lab-3 Task

Hideyuki Shibuki, Kotaro Sakamoto, Madoka Ishioroshi, Yoshinobu Kano, Teruko Mitamura, Tatsunori Mori, Noriko Kando
2017 NTCIR Conference on Evaluation of Information Access Technologies  
The NTCIR-13 QA Lab-3 task aims at the real-world complex Question Answering (QA) technologies using Japanese university entrance exams and their English translation on the subject of "World history". QA Lab-3 has three end-toend tasks for multiple-choice, term and essay questions. The essay task has three subtasks of extraction, summarization and evaluation-method. There were 85 submissions from 13 teams in total. We describe the used data, formal run results, and comparison between human
more » ... and automatic evaluation scores for essay questions.
dblp:conf/ntcir/ShibukiSIKMMK17 fatcat:su6cvqoex5foxeqpneijhfrviy