Speak4IT: Multimodal interaction in the wild

Michael Johnston, Patrick Ehlen
2010 2010 IEEE Spoken Language Technology Workshop  
Speak4it SM is a consumer-oriented mobile search application that leverages multimodal input and output to allow users to search for and act on local business information. In addition to specifying queries by voice (e.g., "bike repair shops near the Golden Gate Bridge") users can combine speech and gesture. For example, "gas stations" + will return the gas stations along the specified route traced on the display. We provide interactive demonstrations of Speak4it on both the iPhone and iPad
more » ... orms and explain the underlying multimodal architecture and challenges of supporting true multimodal interaction in a deployed mobile service.
doi:10.1109/slt.2010.5700840 dblp:conf/slt/JohnstonE10 fatcat:yyxsqhcepnb6jcycxy36637lsq