Region Similarity Representation Learning [article]

Tete Xiao, Colorado J Reed, Xiaolong Wang, Kurt Keutzer, Trevor Darrell
<span title="2021-08-18">2021</span> <i > arXiv </i> &nbsp; <span class="release-stage" >pre-print</span>
We present Region Similarity Representation Learning (ReSim), a new approach to self-supervised representation learning for localization-based tasks such as object detection and segmentation. While existing work has largely focused on solely learning global representations for an entire image, ReSim learns both regional representations for localization as well as semantic image-level representations. ReSim operates by sliding a fixed-sized window across the overlapping area between two views
more &raquo; ... g., image crops), aligning these areas with their corresponding convolutional feature map regions, and then maximizing the feature similarity across views. As a result, ReSim learns spatially and semantically consistent feature representation throughout the convolutional feature maps of a neural network. A shift or scale of an image region, e.g., a shift or scale of an object, has a corresponding change in the feature maps; this allows downstream tasks to leverage these representations for localization. Through object detection, instance segmentation, and dense pose estimation experiments, we illustrate how ReSim learns representations which significantly improve the localization and classification performance compared to a competitive MoCo-v2 baseline: +2.7 AP^bb_75 VOC, +1.1 AP^bb_75 COCO, and +1.9 AP^mk Cityscapes. Code and pre-trained models are released at:
<span class="external-identifiers"> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2103.12902v2">arXiv:2103.12902v2</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/x3j4gwqdbrbormhpw557t33ytu">fatcat:x3j4gwqdbrbormhpw557t33ytu</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20210826075514/https://arxiv.org/pdf/2103.12902v2.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/4f/8a/4f8ac76becef4536472b4b98e486c52bf19700d4.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2103.12902v2" title="arxiv.org access"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> arxiv.org </button> </a>