Vidiam: Corpus-based Development of a Dialogue Manager for Multimodal Question Answering [chapter]

Boris van Schooten, Rieks op den Akker
2011 Interactive Multi-modal Question-Answering  
In this chapter we describe the Vidiam project, which concerns the development of a dialogue management system for multi-modal question answering dialogues as it was carried out in the IMIX project. The approach that was followed is data-driven, that is, corpus-based. Since research in Question Answering Dialog for multi-modal information retrieval is still new, no suitable corpora were available to base a system on. We report on the collection and analysis of three QA dialogue corpora,
more » ... g textual followup utterances, multimodal follow-up questions, and speech dialogues. Based on the data, we created a dialogue act typology which helps translate user utterances to practical interactive QA strategies. We then report how we built and evaluated the dialogue manager and its components: dialogue act recognition, interactive QA strategy handling, reference resolution, and multimodal fusion, using off-line analyses of the corpus data.
doi:10.1007/978-3-642-17525-1_3 dblp:series/tanlp/SchootenA11 fatcat:6b7h3vzedfewhiixw3l5cfo5ze