Probabilistic Matrix Factorization With Semantic And Visual Neighborhoods For Image Tag Completion

Dimitrios Rafailidis
2015 Proceedings of the 5th ACM on International Conference on Multimedia Retrieval - ICMR '15  
We present an image tag completion method, namely PMF-SVN, where the key idea is to exploit images' Semantically and Visually similar Neighborhoods (SVNs) in the learning process of a Probabilistic Matrix Factorization (PMF) framework. We propose a two-step SVN formation algorithm that can generate an image set with the images being both visually and semantically similar. Furthermore, we introduce an efficient way to incorporate the formed SVNs into the learning process of PMF, under the
more » ... int that the latent features of each image are averaged by the features of the images that belong to its SVN. In our experiments with benchmark datasets, we show that the proposed PMF-SVN method outperforms competitive baselines, in terms of completion accuracy, by efficiently capturing the semantical and visual associations between images and tags in SVNs.
doi:10.1145/2671188.2749333 dblp:conf/mir/Rafailidis15 fatcat:brktsovlpzgqflkcqqre7dl2xe