Discovering multirelational structure in social media streams

Yu-Ru Lin, Hari Sundaram, Munmun De Choudhury, Aisling Kelliher
2012 ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP)  
________________________________________________________________________ In this article, we present a novel algorithm to discover multi-relational structures from social media streams. A media item such as a photograph exists as part of a meaningful inter-relationship amongst several attributes includingtime, visual content, users, and actions. Discovery of such relational structures enables us to understand the semantics of human activity and has applications in content organization,
more » ... ation algorithms, and exploratory social network analysis. We are proposing a novel non-negative matrix factorization framework to characterize relational structures of group photo streams. The factorization incorporates image content features and contextual information. The idea is to consider a cluster as having similar relational patternseach cluster consists of photos relating to similar content or context. Relations represent different aspects of the photo stream data, including visual content, associated tags, photo owners, and post times. The extracted structures minimize the mutual information of the predicted joint distribution. We also introduce a relational modularity function to determine the structure cost penalty, and hence determine the number of clusters. Extensive experiments on a large Flickr dataset suggest that our approach is able to extract meaningful relational patterns from group photo streams. We evaluate the utility of the discovered structures through a tag prediction task and through a user study. Our results show that our method based on relational structures, outperforms baseline methods, including feature and tag frequency based techniques, by 35%-420%. We have conducted a qualitative user study to evaluate the benefits of our framework in exploring group photo streams. The study indicates that users found the extracted clustering results clearly represent major themes in a group; the clustering results not only reflect how users describe the group data but often lead the users to discover the evolution of the group activity.
doi:10.1145/2071396.2071400 fatcat:5hmyvg53tvaz3db2c23how7kaq