Multi-modal dialogue system with sign language capabilities

Marek Hrúz, Pavel Campr, Zdenek Krňoul, Milos Železný, Oya Aran, Pinar Santemiz
2011 The proceedings of the 13th international ACM SIGACCESS conference on Computers and accessibility - ASSETS '11  
This paper presents the design of a multimodal sign-language -enabled dialogue system. Its functionality was tested on a prototype of an information kiosk for the deaf people providing information about train connections. We use an automatic computer-vision-based sign language recognition, automatic speech recognition and touchscreen as input modalities. The outputs are shown on a screen displaying 3D signing avatar and on a touchscreen displaying graphical user interface. The information kiosk
more » ... can be used both by hearing users and deaf users in several languages. We focus on description of sign language input and output modality.
doi:10.1145/2049536.2049599 dblp:conf/assets/HruzCKZAS11 fatcat:fheaty2vibgpfmydiwnlxjftve