Multimodal analysis of speech prosody and upper body gestures using hidden semi-Markov models

Elif Bozkurt, Shahriar Asta, Serkan Ozkul, Yucel Yemez, Engin Erzin
<span title="">2013</span> <i title="IEEE"> <a target="_blank" rel="noopener" href="" style="color: black;">2013 IEEE International Conference on Acoustics, Speech and Signal Processing</a> </i> &nbsp;
Gesticulation is an essential component of face-to-face communication, and it contributes significantly to the natural and affective perception of human-to-human communication. In this work we investigate a new multimodal analysis framework to model relationships between intonational and gesture phrases using the hidden semi-Markov models (HSMMs). The HSMM framework effectively associates longer duration gesture phrases to shorter duration prosody clusters, while maintaining realistic gesture
more &raquo; ... rase duration statistics. We evaluate the multimodal analysis framework by generating speech prosody driven gesture animation, and employing both subjective and objective metrics.
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="">doi:10.1109/icassp.2013.6638339</a> <a target="_blank" rel="external noopener" href="">dblp:conf/icassp/BozkurtAOYE13</a> <a target="_blank" rel="external noopener" href="">fatcat:cklrs3rfzzb67fgx3lmhrimh44</a> </span>
<a target="_blank" rel="noopener" href="" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href=""> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> </button> </a>