Learning Multi-modal Similarity [article]

Brian McFee, Gert Lanckriet
<span title="2010-08-30">2010</span> <i > arXiv </i> &nbsp; <span class="release-stage" >pre-print</span>
In many applications involving multi-media data, the definition of similarity between items is integral to several key tasks, e.g., nearest-neighbor retrieval, classification, and recommendation. Data in such regimes typically exhibits multiple modalities, such as acoustic and visual content of video. Integrating such heterogeneous data to form a holistic similarity space is therefore a key challenge to be overcome in many real-world applications. We present a novel multiple kernel learning
more &raquo; ... nique for integrating heterogeneous data into a single, unified similarity space. Our algorithm learns an optimal ensemble of kernel transfor- mations which conform to measurements of human perceptual similarity, as expressed by relative comparisons. To cope with the ubiquitous problems of subjectivity and inconsistency in multi- media similarity, we develop graph-based techniques to filter similarity measurements, resulting in a simplified and robust training procedure.
<span class="external-identifiers"> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/1008.5163v1">arXiv:1008.5163v1</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/demnetxmlfadtit6mxbsxfsniy">fatcat:demnetxmlfadtit6mxbsxfsniy</a> </span>
<a target="_blank" rel="noopener" href="https://archive.org/download/arxiv-1008.5163/1008.5163.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> File Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/d0/63/d06346fe3aae35b65117b5fd7611093d133b7f06.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/1008.5163v1" title="arxiv.org access"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> arxiv.org </button> </a>