Action Recognition Using Mined Hierarchical Compound Features
IEEE Transactions on Pattern Analysis and Machine Intelligence
The field of Action Recognition has seen a large increase in activity in recent years. Much of the progress has been through incorporating ideas from single frame object recognition and adapting them for temporal based action recognition. Inspired by the success of interest points in the 2D spatial domain, their 3D (space-time) counterparts typically form the basic components used to describe actions, and in action recognition, the features used are often engineered to fire sparsely. This is to
... parsely. This is to ensure the problem is tractable; however, this can sacrifice recognition accuracy as it cannot be assumed that the optimum features in terms of class discrimination are obtained from this approach. In contrast, we propose to initially use an over complete set of simple 2D corners in both space and time. These are grouped spatially and temporally using a hierarchical process, with an increasing search area. At each stage of the hierarchy, the most distinctive and descriptive features are learnt efficiently through data mining. This allows large amounts of data to be searched for frequently reoccurring patterns of features. At each level of the hierarchy, the mined compound features become more complex, discriminative and sparse. This results in fast, accurate recognition with real-time performance on high resolution video. As the compound features are constructed and selected based upon their ability to discriminate, their speed and accuracy increase at each level of the hierarchy. The approach is tested on four state-of-the-art datasets, the popular KTH dataset to provide a comparison with other state-of-the-art approaches, the Multi-KTH dataset to illustrate performance at simultaneous multiaction classification, despite no explicit localization information provided during training. Finally, the recent Hollywood and Hollywood2 datasets, provide challenging complex actions taken from commercial movie sequences. For all four datasets, the proposed hierarchical approach out performs all other methods reported thus far in the literature, and can achieve real-time operation.