Training combination strategy of multi-stream fused hidden Markov model for audio-visual affect recognition

Zhihong Zeng, Yuxiao Hu, Ming Liu, Yun Fu, Thomas S. Huang
2006 Proceedings of the 14th annual ACM international conference on Multimedia - MULTIMEDIA '06  
To simulate the human ability to assess affects, an automatic affect recognition system should make use of multi-sensor information. In the framework of multi-stream fused hidden Markov model (MFHMM), we present a training combination strategy towards audio-visual affect recognition. Different from the weighting combination scheme, our approach is able to use a variety of learning methods to obtain a robust multi-stream fusion result. We evaluate our approach in personal-independent recognition
more » ... of 11 affective states from 20 subjects. The experimental results suggest that MFHMM outperforms IHMM which assumes the independence among streams, and the training combination strategy has the superiority over the weighting combination under clean and varying audio channel noise condition.
doi:10.1145/1180639.1180661 dblp:conf/mm/ZengHLFH06 fatcat:2gk5hgkjszht7eknl3twfx4kuy