Emotion recognition from embedded bodily expressions and speech during dyadic interactions

Philipp M. Muller, Sikandar Amin, Prateek Verma, Mykhaylo Andriluka, Andreas Bulling
2015 2015 International Conference on Affective Computing and Intelligent Interaction (ACII)  
Previous work on emotion recognition from bodily expressions focused on analysing such expressions in isolation, of individuals or in controlled settings, from a single camera view, or required intrusive motion tracking equipment. We study the problem of emotion recognition from bodily expressions and speech during dyadic (person-person) interactions in a real kitchen instrumented with ambient cameras and microphones. We specifically focus on bodily expressions that are embedded in regular
more » ... actions and background activities and recorded without human augmentation to increase naturalness of the expressions. We present a human-validated dataset that contains 224 high-resolution, multi-view video clips and audio recordings of emotionally charged interactions between eight couples of actors. The dataset is fully annotated with categorical labels for four basic emotions (anger, happiness, sadness, and surprise) and continuous labels for valence, activation, power, and anticipation provided by five annotators for each actor. We evaluate vision and audio-based emotion recognition using dense trajectories and a standard audio pipeline and provide insights into the importance of different body parts and audio features for emotion recognition.
doi:10.1109/acii.2015.7344640 dblp:conf/acii/MullerAVAB15 fatcat:zkphthbeezffbkouiogua6sy7y