40,078 Hits in 3.8 sec

Continual learning in cross-modal retrieval [article]

Kai Wang, Luis Herranz, Joost van de Weijer
2021 arXiv   pre-print
In this paper, we propose a combination of both problems into a continual cross-modal retrieval setting, where we study how the catastrophic interference caused by new tasks impacts the embedding spaces  ...  and their cross-modal alignment required for effective retrieval.  ...  Figure 3 . 3 Types of pairs in continual cross-modal retrieval: (a) available in joint training, and (b) available in continual learning, i.e. without cross-task negative pairs (CTNP).  ... 
arXiv:2104.06806v2 fatcat:qlgyuuxcnbhyjes5n36cm7kfym

Cross-Modal Hamming Hashing [chapter]

Yue Cao, Bin Liu, Mingsheng Long, Jianmin Wang
2018 Lecture Notes in Computer Science  
Cross-modal hashing enables similarity retrieval across different content modalities, such as searching relevant images in response to text queries.  ...  This work presents Cross-Modal Hamming Hashing (CMHH), a novel deep cross-modal hashing approach that generates compact and highly concentrated hash codes to enable efficient and effective Hamming space  ...  This paper presents Cross-Modal Hamming Hashing (CMHH), a unified deep learning framework for cross-modal Hamming space retrieval, as shown in Fig. 2 .  ... 
doi:10.1007/978-3-030-01246-5_13 fatcat:42mkbkebrvfhrnmctbk5p65npu

Transitive Hashing Network for Heterogeneous Multimedia Retrieval [article]

Zhangjie Cao, Mingsheng Long, Qiang Yang
2016 arXiv   pre-print
Cross-modal hashing enables efficient retrieval from database of one modality in response to a query of another modality.  ...  Existing work on cross-modal hashing assumes heterogeneous relationship across modalities for hash function learning.  ...  We use the mini-batch stochastic gradient descent (SGD) with 0.9 momentum and the learning rate strategy in Caffe, cross-validate learning rate from 10 −5 to 10 −1 with a multiplicative step-size 10 1/  ... 
arXiv:1608.04307v1 fatcat:l2xakppke5b5vgwcgs7wwliufq

Correlation Hashing Network for Efficient Cross-Modal Retrieval [article]

Yue Cao, Mingsheng Long, Jianmin Wang, Philip S. Yu
2017 arXiv   pre-print
Extensive empirical study shows that CHN yields state of the art cross-modal retrieval performance on standard benchmarks.  ...  Cross-modal hashing improves the quality of hash coding by exploiting semantic correlations across different modalities.  ...  These results verify that CHN is able to learn high-quality hash codes for effective cross-modal retrieval.  ... 
arXiv:1602.06697v2 fatcat:wemcafkkojbvppdkclyj5knglq

Emotion Embedding Spaces for Matching Music to Stories

Minz Won, Justin Salamon, Nicholas J. Bryan, Gautham Mysore, Xavier Serra
2021 Zenodo  
Our experiments show that by leveraging these embedding spaces, we are able to successfully bridge the gap between modalities to facilitate cross modal retrieval.  ...  ., books), use multiple sentences as input queries, and automatically retrieve matching music. We formalize this task as a cross-modal text-to-music retrieval problem.  ...  cross-modal retrieval.  ... 
doi:10.5281/zenodo.5624482 fatcat:uqlm3s5korb5rm2ybkbvr42qpi

Sequential Learning for Cross-Modal Retrieval

Ge Song, Xiaoyang Tan
2019 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)  
Inspired by the recent achievement in the field of cognition mechanism on how the human brain acquires knowledge, we propose a new sequential learning method for cross-modal retrieval.  ...  Cross-modal retrieval has attracted increasing attention with the rapid growth of multimodal data, but its learning paradigm under changing environment is less studied.  ...  Related Work Cross-modal retrieval. Cross-modal learning approaches [2, 9, 5, 19, 31, 16] can roughly be divided into continual-value learning method and hashing method.  ... 
doi:10.1109/iccvw.2019.00554 dblp:conf/iccvw/SongT19 fatcat:wq7czx7byvhsngmhhwzofsvde4

Modality-Dependent Cross-Modal Retrieval Based on Graph Regularization

Guanhua Wang, Hua Ji, Dexin Kong, Na Zhang
2020 Mobile Information Systems  
In order to fully exploit the potential correlation of different modalities, we propose a cross-modal retrieval framework based on graph regularization and modality dependence (GRMD).  ...  Nowadays, the heterogeneity gap of different modalities is the key problem for cross-modal retrieval.  ...  cross-modal retrieval based on graph regularization in I2T. datasets.  ... 
doi:10.1155/2020/4164692 fatcat:2ku7xp5x65bkhggvu7x7skspja

A Cross-Media Retrieval Method Based on Semisupervised Learning and Alternate Optimization

Junzheng Li, Wei Zhu, Yanchun Yang, Xiyuan Zheng
2021 Mobile Information Systems  
The most difficult task for cross-media retrieval lies in the potential correlation between different modalities data and how to overcome the semantic gap.  ...  This paper proposes a cross-media retrieval method based on semisupervised learning and alternate optimization (SMDCR) to overcome the abovementioned difficulties, thereby improving the retrieval accuracy  ...  modalities data in cross-media retrieval.  ... 
doi:10.1155/2021/9947644 doaj:90a40b8f33a34c6fa8e02d33c8713613 fatcat:3hfjrkeerfa3tgbtlmwd3o6qd4

Deep Multi-level Semantic Hashing for Cross-modal Retrieval

Zhenyan Ji, Weina Yao, Wei Wei, Houbing Song, Huaiyu Pi
2019 IEEE Access  
INDEX TERMS Cross-modal retrieval, deep learning, hashing method, multi-label learning.  ...  Due to its efficiency on storage and computing, hashing-based methods are broadly used for large scale cross-modal retrieval.  ...  Cross-modal retrieval, also called cross-media retrieval, models the relationship among different modalities.  ... 
doi:10.1109/access.2019.2899536 fatcat:xynopqlgyfhe3ef6su55zqczim

Using Deep Cross Modal Hashing and Error Correcting Codes for Improving the Efficiency of Attribute Guided Facial Image Retrieval [article]

Veeru Talreja, Fariborz Taherkhani, Matthew C. Valenti, Nasser M. Nasrabadi
2019 arXiv   pre-print
In this paper, we propose a novel Error-Corrected Deep Cross Modal Hashing (CMH-ECC) method which uses a bitmap specifying the presence of certain facial attributes as an input query to retrieve relevant  ...  In this architecture, we generate compact hash codes using an end-to-end deep learning module, which effectively captures the inherent relationships between the face and attribute modality.  ...  Recently, application of deep learning to hash methods for uni-modal image retrieval [1, 2] and cross-modal retrieval [3, 4] have shown that end-to-end learning of feature extraction and hash coding  ... 
arXiv:1902.04139v1 fatcat:ychlkoe6yvaqfppi4xin5afysy

Deep Learning Techniques for Future Intelligent Cross-Media Retrieval [article]

Sadaqat ur Rehman, Muhammad Waqas, Shanshan Tu, Anis Koubaa, Obaid ur Rehman, Jawad Ahmad, Muhammad Hanif, Zhu Han
2020 arXiv   pre-print
In this paper, we provide a novel taxonomy according to the challenges faced by multi-modal deep learning approaches in solving cross-media retrieval, namely: representation, alignment, and translation  ...  Then, we present some well-known cross-media datasets used for retrieval, considering the importance of these datasets in the context in of deep learning based cross-media retrieval approaches.  ...  Representations Data representations in cross-modal retrieval has always been a difficult task in deep learning.  ... 
arXiv:2008.01191v1 fatcat:t63bg55w2vdqjcprzaaidrmprq

Cross-Model Hashing Retrieval Based on Deep Residual Network

Zhiyi Li, Xiaomian Xu, Du Zhang, Peng Zhang
2021 Computer systems science and engineering  
In the process of mapping, the distance measurement of the original distance measurement and the common feature space are kept unchanged as far as possible to improve the accuracy of Cross-Modal Retrieval  ...  In the era of big data rich in We Media, the single mode retrieval system has been unable to meet people's demand for information retrieval.  ...  Acknowledgement: This paper would like to thank all the authors cited in the reference for their contributions to this field.  ... 
doi:10.32604/csse.2021.014563 fatcat:maw3votu7bdl5csccvqniby65i

Unsupervised Generative Adversarial Cross-modal Hashing [article]

Jian Zhang, Yuxin Peng, Mingkuan Yuan
2017 arXiv   pre-print
learning to exploit the underlying manifold structure of cross-modal data.  ...  helpful to capture meaningful nearest neighbors of different modalities for cross-modal retrieval.  ...  Therefore, cross-modal hashing has been proposed to meet this kind of retrieval demands in large scale cross-modal databases.  ... 
arXiv:1712.00358v1 fatcat:sxe6tprexzfy3h6iknjywwrf7u

Composite Correlation Quantization for Efficient Multimodal Retrieval

Mingsheng Long, Yue Cao, Jianmin Wang, Philip S. Yu
2016 Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval - SIGIR '16  
While hashing methods have shown great potential in achieving this goal, current attempts generally fail to learn isomorphic hash codes in a seamless scheme, that is, they embed multiple modalities in  ...  a continuous isomorphic space and separately threshold embeddings into binary codes, which incurs substantial loss of retrieval accuracy.  ...  The first two tasks are intra-modal retrieval and the last two tasks are cross-modal retrieval.  ... 
doi:10.1145/2911451.2911493 dblp:conf/sigir/LongCWY16 fatcat:y7omekozmjghzjztzn3wtij3cq

Query by Video: Cross-modal Music Retrieval

Bochen Li, Aparna Kumar
2019 Zenodo  
Cross-modal retrieval learns the relationship between the two types of data in a common space so that an input from one modality can retrieve data from a different modality.  ...  To retrieve music for an input video, the trained model ranks tracks in the music database by cross-modal distances to the query video.  ...  Cross-modal Audio-Visual Retrieval Cross-modal retrieval has received increasing attention in the recent years.  ... 
doi:10.5281/zenodo.3527881 fatcat:cwwcc6objbca7puhyt5rbxln6u
« Previous Showing results 1 — 15 out of 40,078 results