Audio-visual integration in multimodal communication

Tsuhan Chen, R.R. Rao
1998 Proceedings of the IEEE  
In this paper, we review recent research that examines audiovisual integration in multimodal communication. The topics include bimodality in human speech, human and automated lip reading, facial animation, lip synchronization, joint audiovideo coding, and bimodal speaker verification. We also study the enabling technologies for these research topics, including automatic facial-feature tracking and audio-to-visual mapping. Recent progress in audio-visual research shows that joint processing of
more » ... dio and video provides advantages that are not available when the audio and video are processed independently.
doi:10.1109/5.664274 fatcat:2wpoj4xcdvbcbddcwttx6cn27q