A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf
.
RePre: Improving Self-Supervised Vision Transformer with Reconstructive Pre-training
[article]
2022
arXiv
pre-print
Recently, self-supervised vision transformers have attracted unprecedented attention for their impressive representation learning ability. However, the dominant method, contrastive learning, mainly relies on an instance discrimination pretext task, which learns a global understanding of the image. This paper incorporates local feature learning into self-supervised vision transformers via Reconstructive Pre-training (RePre). Our RePre extends contrastive frameworks by adding a branch for
arXiv:2201.06857v2
fatcat:alqu2db7xngmphbc23w7gwoobi