Integrating multi-stage depth-induced contextual information for human action recognition and localization

Bingbing Ni, Yong Pei, Zhujin Liang, Liang Lin, Pierre Moulin
2013 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG)  
Human action recognition and localization is a challenging vision task with promising applications. To tackle this problem, recently developed commodity depth sensor (e.g., Microsoft Kinect) has opened up new opportunities with several developed human motion features based on depth image for action representation. However, how depth information can be effectively adopted in the middle or high level representation in action detection, in particular, the depth induced three dimensional contextual
more » ... information for modeling interactions between human-human, human-object and human-surroundings has yet been explored. In this paper, we propose a novel action recognition and localization framework which effectively fuses depth-induced contextual information from different levels of the processing pipeline for understanding various interactions. First, depth image is combined with grayscale image for more robust human subject and object detection. Second, three dimensional spatial and temporal relationship among human subjects or objects is represented based on the combination of grayscale and depth images. Third, depth information is further utilized to represent different types of indoor scenes. Finally, we fuse these multiple stage depth-induced contextual information to yield an unified action detection framework. Extensive experiments on a challenging grayscale + depth human action detection benchmark database demonstrate the effectiveness of the depth-induced contextual information and the high detection accuracy of the proposed framework.
doi:10.1109/fg.2013.6553756 dblp:conf/fgr/NiPLLM13 fatcat:jnxm2wtbnfgqbmu6x3dphjliju