Human Activity Recognition Using Semi-supervised Multi-modal DEC for Instagram Data [chapter]

Dongmin Kim, Sumin Han, Heesuk Son, Dongman Lee
2020 Lecture Notes in Computer Science  
Human Activity Recognition (HAR) using social media provides a solid basis for a variety of context-aware applications. Existing HAR approaches have adopted supervised machine learning algorithms using texts and their meta-data such as time, venue, and keywords. However, their recognition accuracy may decrease when applied to imagesharing social media where users mostly describe their daily activities and thoughts using both texts and images. In this paper, we propose a semi-supervised
more » ... al deep embedding clustering method to recognize human activities on Instagram. Our proposed method learns multi-modal feature representations by alternating a supervised learning phase and an unsupervised learning phase. By utilizing a large number of unlabeled data, it learns a more generalized feature distribution for each HAR class and avoids overfitting to limited labeled data. Evaluation results show that leveraging multi-modality and unlabeled data is effective for HAR and our method outperforms existing approaches.
doi:10.1007/978-3-030-47426-3_67 fatcat:wqnltmtwencbrdvamml5cm4fqe