Weakly Supervised Visual Dictionary Learning by Harnessing Image Attributes

Yue Gao, Rongrong Ji, Wei Liu, Qionghai Dai, Gang Hua
2014 IEEE Transactions on Image Processing  
Bag-of-features (BoFs) representation has been extensively applied to deal with various computer vision applications. To extract discriminative and descriptive BoF, one important step is to learn a good dictionary to minimize the quantization loss between local features and codewords. While most existing visual dictionary learning approaches are engaged with unsupervised feature quantization, the latest trend has turned to supervised learning by harnessing the semantic labels of images or
more » ... s. However, such labels are typically too expensive to acquire, which restricts the scalability of supervised dictionary learning approaches. In this paper, we propose to leverage image attributes to weakly supervise the dictionary learning procedure without requiring any actual labels. As a key contribution, our approach establishes a generative hidden Markov random field (HMRF), which models the quantized codewords as the observed states and the image attributes as the hidden states, respectively. Dictionary learning is then performed by supervised grouping the observed states, where the supervised information is stemmed from the hidden states of the HMRF. In such a way, the proposed dictionary learning approach incorporates the image attributes to learn a semanticpreserving BoF representation without any genuine supervision. Experiments in large-scale image retrieval and classification tasks corroborate that our approach significantly outperforms the state-of-the-art unsupervised dictionary learning approaches. Index Terms-Bag-of-features, visual dictionary, image attribute, weakly supervised learning, hidden Markov random field, image classification, image search.
doi:10.1109/tip.2014.2364536 pmid:25361504 fatcat:vfuulvkvbvd6pjpn3libbzria4