A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf
.
Latent Semantic Analysis for Multimodal User Input With Speech and Gestures
2014
IEEE/ACM Transactions on Audio Speech and Language Processing
This paper describes our work in semantic interpretation of a "multimodal language" with speech and gestures using latent semantic analysis (LSA). Our aim is to infer the domain-specific informational goal of multimodal inputs. The informational goal is characterized by lexical terms used in the spoken modality, partial semantics of gestures in the pen modality, as well as term co-occurrence patterns across modalities, leading to "multimodal terms." We designed and collected a multimodal corpus
doi:10.1109/taslp.2013.2294586
fatcat:53a4qg4isjegnplrqm7ygjqkiu