Continuous ultrasound based tongue movement video synthesis from speech

Jianrong Wang, Yalong Yang, Jianguo Wei, Ju Zhang
2016 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)  
The movement of tongue plays an important role in pronunciation. Visualizing the movement of tongue can improve speech intelligibility and also helps learning a second language. However, hardly any research has been investigated for this topic. In this paper, a framework to synthesize continuous ultrasound tongue movement video from speech is presented. Two different mapping methods are introduced as the most important parts of the framework. The objective evaluation and subjective opinions
more » ... that the Gaussian Mixture Model (GMM) based method has a better result for synthesizing static image and Vector Quantization (VQ) based method produces more stable continuous video. Meanwhile, the participants of evaluation state that the results of both methods are visual-understandable.
doi:10.1109/icassp.2016.7471970 dblp:conf/icassp/WangYWZ16 fatcat:5fmx4qdph5apdo7hrzm3f4uo6i