From Part to Whole

Daiqian Ma, Feng Gao, Yan Bai, Yihang Lou, Shiqi Wang, Tiejun Huang, Ling-Yu Duan
2017 Proceedings of the 2017 ACM on Multimedia Conference - MM '17  
Compared with normal modalities, the representations of paintings are much more complex due to its large intra-class and small inter-class variation. This poses more di culties in the task of authorship identi cation. In this paper, we propose a multi-task multi-range (MTMR) representation framework and try to resolve this issue in two ways. First, we investigate how to improve the representation through multi-task learning. Speci cally, we attempt to optimize authorship identi cation with
more » ... y correlated identi cation tasks such as style, genre and date. Second, in order to make the representation more comprehensive and reduce the information loss from image scaling, we propose a multi-range structure which is composed of local, regional and global representations. Experiments on the two most representative large-scale painting datasets, Rijksmuseum Challenge and Wikiart, have shown that our method signi cantly outperforms the existing methods. To give better understanding and provide more e ective predictions, we utilize random forest as the feature ranking method to analyze the importance of di erent features and apply external knowledge matching to further examine the predictions. Moreover, the framework's effects of identifying the authorship are visualized on the paintings' artist-characteristic regions and t-SNE is further applied to perform artist-based cluster analysis. Extensive validation has demonstrated that the proposed framework yields superior performance in the chanllenging task of painting authorship identi cation. KEYWORDS Painting; multi-task learning; multi-range representation; external knowledge matching; random forest; * Ling-Yu Duan is the corresponding author. Daiqian Ma and Feng Gao are joint rst authors.
doi:10.1145/3123266.3123325 dblp:conf/mm/MaGBLWHD17 fatcat:2mqh3gybavb47jkfbguagdenly