STEP: Spatial Temporal Graph Convolutional Networks for Emotion Perception from Gaits [article]

Uttaran Bhattacharya and Trisha Mittal and Rohan Chandra and Tanmay Randhavane and Aniket Bera and Dinesh Manocha
2019 arXiv   pre-print
We present a novel classifier network called STEP, to classify perceived human emotion from gaits, based on a Spatial Temporal Graph Convolutional Network (ST-GCN) architecture. Given an RGB video of an individual walking, our formulation implicitly exploits the gait features to classify the emotional state of the human into one of four emotions: happy, sad, angry, or neutral. We use hundreds of annotated real-world gait videos and augment them with thousands of annotated synthetic gaits
more » ... ed using a novel generative network called STEP-Gen, built on an ST-GCN based Conditional Variational Autoencoder (CVAE). We incorporate a novel push-pull regularization loss in the CVAE formulation of STEP-Gen to generate realistic gaits and improve the classification accuracy of STEP. We also release a novel dataset (E-Gait), which consists of 2,177 human gaits annotated with perceived emotions along with thousands of synthetic gaits. In practice, STEP can learn the affective features and exhibits classification accuracy of 89 30
arXiv:1910.12906v1 fatcat:lkmwd5kidfcmfbpfbde6rrkq5u