Multimodal conversational interaction with a humanoid robot

Adam Csapo, Emer Gilmartin, Jonathan Grizou, JingGuang Han, Raveesh Meena, Dimitra Anastasiou, Kristiina Jokinen, Graham Wilcock
2012 2012 IEEE 3rd International Conference on Cognitive Infocommunications (CogInfoCom)  
The paper presents a multimodal conversational interaction system for the Nao humanoid robot. The system was developed at the 8th International Summer Workshop on Multimodal Interfaces, Metz, 2012. We implemented WikiTalk, an existing spoken dialogue system for open-domain conversations, on Nao. This greatly extended the robot's interaction capabilities by enabling Nao to talk about an unlimited range of topics. In addition to speech interaction, we developed a wide range of multimodal
more » ... multimodal interactive behaviours by the robot, including facetracking, nodding, communicative gesturing, proximity detection and tactile interrupts. We made video recordings of user interactions and used questionnaires to evaluate the system. We further extended the robot's capabilities by linking Nao with Kinect. Index Terms-human-robot interaction, spoken dialogue systems, communicative gesturing.
doi:10.1109/coginfocom.2012.6421935 fatcat:bscrv5zfsbhdnjm2cvs4gzxwxq