Multimodal analysis of speech prosody and upper body gestures using hidden semi-Markov models

Elif Bozkurt, Shahriar Asta, Serkan Ozkul, Yucel Yemez, Engin Erzin
<span title="">2013</span> <i title="IEEE"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/rc5jnc4ldvhs3dswicq5wk3vsq" style="color: black;">2013 IEEE International Conference on Acoustics, Speech and Signal Processing</a> </i> &nbsp;
Gesticulation is an essential component of face-to-face communication, and it contributes significantly to the natural and affective perception of human-to-human communication. In this work we investigate a new multimodal analysis framework to model relationships between intonational and gesture phrases using the hidden semi-Markov models (HSMMs). The HSMM framework effectively associates longer duration gesture phrases to shorter duration prosody clusters, while maintaining realistic gesture
more &raquo; ... rase duration statistics. We evaluate the multimodal analysis framework by generating speech prosody driven gesture animation, and employing both subjective and objective metrics.
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1109/icassp.2013.6638339">doi:10.1109/icassp.2013.6638339</a> <a target="_blank" rel="external noopener" href="https://dblp.org/rec/conf/icassp/BozkurtAOYE13.html">dblp:conf/icassp/BozkurtAOYE13</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/cklrs3rfzzb67fgx3lmhrimh44">fatcat:cklrs3rfzzb67fgx3lmhrimh44</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20160325063656/http://astas.co.uk/papers/ub_icassp13_14march.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/d3/20/d320093124963a4d7bcbed17ddbe45210b20c4eb.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1109/icassp.2013.6638339"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> ieee.com </button> </a>