Multimodal fusion

Pilar Manchón Portillo, Guillermo Pérez García, Gabriel Amores Carredano
2006 Proceedings of the 8th international conference on Multimodal interfaces - ICMI '06  
This is a new hybrid fusion strategy based primarily on the implementation of two former and differentiated approaches to multimodal fusion [11] in multimodal dialogue systems. Both approaches, their predecessors and their respective advantages and disadvantages will be described in order to illustrate how the new strategy merges them into a more solid and coherent solution. The first strategy was largely based on Johnston´s approach [5] and implies the inclusion of multimodal grammar entries
more » ... d temporal constraints. The second approach implied the fusion of information coming from different channels at dialogue level. The new hybrid strategy hereby described requires the inclusion of multimodal grammar entries and temporal constraints plus the additional information at dialogue level utilized in the second strategy. Within this new approach therefore, the fusion process will be initiated at grammar level and will be culminated at dialogue level.
doi:10.1145/1180995.1181061 dblp:conf/icmi/PortilloGC06 fatcat:jw32xov3wvaazmhgfv2ewu4xhq