Internet Archive Scholar

Learning Cross-Modal Deep Embeddings for Multi-Object Image Retrieval using Text and Sketch [article]

Sounak Dey and Anjan Dutta and Suman K. Ghosh and Ernest Valveny and Josep Lladós and Umapada Pal
2018 arXiv   pre-print

Preserved Fulltext

fulltext thumbnail
Web Archive Capture PDF (10.5 MB)
https://web.archive.org/web/20201009103545/https://arxiv.org/pdf/1804.10819v1.pdf

A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL. The file type is application/pdf.

In this work we introduce a cross modal image retrieval system that allows both text and sketch as input modalities for the query. A cross-modal deep network architecture is formulated to jointly model the sketch and text input modalities as well as the the image output modality, learning a common embedding between text and images and between sketches and images. In addition, an attention model is used to selectively focus the attention on the different objects of the image, allowing for
more » ... al with multiple objects in the query. Experiments show that the proposed method performs the best in both single and multiple object image retrieval in standard datasets.
arXiv:1804.10819v1 fatcat:dc4o5ocajfddfm6652puqy2ize
Citation

Sounak Dey and Anjan Dutta and Suman K. Ghosh and Ernest Valveny and Josep Lladós and Umapada Pal. "Learning Cross-Modal Deep Embeddings for Multi-Object Image Retrieval using Text and Sketch." arXiv (2018)