Cross-Modal Hashing by lp -Norm Multiple Subgraph Combination

Dongxiao Ren, Junwei Huang, Zhonghua Wang, Fang Lu
2021 IEEE Access  
With the explosion of multi-modal Web data, effective and efficient techniques are in urgent need for cross-modal data retrieval with relevant semantics. Among all the possible solutions, the hashing techniques provide compact and measurable binary representation, thus gain much attention in related research domain. To better deal with diversified real world data, we propose MSC, a novel cross-modal hashing approach based on the generalized l p -norm Multiple Subgraph Combination. Specifically,
more » ... by jointly considering the content similarity, the correspondence and other weak correlation among cross-modal documents, we build the intra-modal similarity with multiple affinity subgraphs, and encode the intermodal correlation with a bipartite subgraph. Then these subgraphs are combined into one multi-modal similarity graph for all the data from heterogeneous modalities, where the weights of multiple intra-modal visual similarity subgraphs are regularized by l p -norm penalty. The optimal hash codes and the combination coefficients are learned simultaneously by efficient alternating optimization. The hash functions for different modalities are learned separately by utilizing nonlinear classification models, encoding the complicated semantic relations among cross-modal data. Experiments on challenging real world datasets demonstrate the advantage of our method over existing approaches. INDEX TERMS Cross-modal hashing, feature combination, information fusion. 19682 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see VOLUME 9, 2021
doi:10.1109/access.2021.3052605 fatcat:budiej6vsvfvtki2pu2uzjudii