CMU Multiple-choice Question Answering System at NTCIR-11 QA-Lab

Di Wang, Leonid Boytsov, Jun Araki, Alkesh Patel, Jeff Gee, Zhengzhong Liu, Eric Nyberg, Teruko Mitamura
2014 NTCIR Conference on Evaluation of Information Access Technologies  
We describe CMU's UIMA-based modular automatic question answering (QA) system. This system answers multiplechoice English questions for the world history entrance exam. Questions are preceded by short descriptions providing a historical context. Given the context and question-specific instructions, we generate verifiable assertions for each answer choice. These assertions are evaluated using several evidencing modules, which assign a plausibility score to each assertion. These scores are then
more » ... gregated to produce the most plausible answer choice. In the NTCIR-11 QALab evaluations, our system achieved 51.6% accuracy on the training set, 47.2% on Phase 1 testing set, and 34.1% on Phase 2 testing set.
dblp:conf/ntcir/WangBAPGLNM14 fatcat:lqx6wauf7zdyfg54wtjcfot2ma