55,223 Hits in 2.6 sec

Self-supervised Representation Learning with Relative Predictive Coding [article]

Yao-Hung Hubert Tsai, Martin Q. Ma, Muqiao Yang, Han Zhao, Louis-Philippe Morency, Ruslan Salakhutdinov
2021 arXiv   pre-print
This paper introduces Relative Predictive Coding (RPC), a new contrastive representation learning objective that maintains a good balance among training stability, minibatch size sensitivity, and downstream  ...  We empirically verify the effectiveness of RPC on benchmark vision and speech self-supervised learning tasks.  ...  RELATED WORK As a subset of unsupervised representation learning, self-supervised representation learning (SSL) adopts self-defined signals as supervision and uses the learned representation for downstream  ... 
arXiv:2103.11275v3 fatcat:o6lqfm4virabjhawyhfpyz22by

InferCode: Self-Supervised Learning of Code Representations by Predicting Subtrees [article]

Nghi D. Q. Bui, Yijun Yu, Lingxiao Jiang
2020 arXiv   pre-print
This paper proposes InferCode to overcome the limitation by adapting the self-supervised learning mechanism to build source code model.  ...  cross-language code search or reused under a transfer learning scheme to continue training the model weights for supervised tasks such as code classification and method name prediction.  ...  Little effort has been invested in the literature to exploit the uses of self-supervised learning for code representation learning.  ... 
arXiv:2012.07023v2 fatcat:jxhfs2a6qfeabehgjvt4bavkfe

Contrastive Predictive Coding for Human Activity Recognition [article]

Harish Haresamudram, Irfan Essa, Thomas Ploetz
2020 arXiv   pre-print
CPC-based pre-training is self-supervised, and the resulting learned representations can be integrated into standard activity chains.  ...  We introduce the Contrastive Predictive Coding (CPC) framework to human activity recognition, which captures the long-term temporal structure of sensor data streams.  ...  SELF-SUPERVISED PRE-TRAINING WITH CONTRASTIVE PREDICTIVE CODING In this paper, we introduce the Contrastive Predictive Coding (CPC) framework to human activity recognition from wearables.  ... 
arXiv:2012.05333v1 fatcat:eti3dtybbbeyhneliephjmtplq

Video Representation Learning by Dense Predictive Coding

Tengda Han, Weidi Xie, Andrew Zisserman
2019 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)  
We make three contributions: First, we introduce the Dense Predictive Coding (DPC) framework for selfsupervised representation learning on videos.  ...  With single stream (RGB only), DPC pretrained representations achieve state-of-the-art self-supervised performance on both UCF101 (75.7% top1 acc) and HMDB51 (35.7% top1 acc), outperforming all previous  ...  DPC with self-supervised learning to further enhance the representation quality.  ... 
doi:10.1109/iccvw.2019.00186 dblp:conf/iccvw/HanXZ19 fatcat:w3rcqd5li5bblo6mal7rvoqo7q

Video Representation Learning by Dense Predictive Coding [article]

Tengda Han, Weidi Xie, Andrew Zisserman
2019 arXiv   pre-print
We make three contributions: First, we introduce the Dense Predictive Coding (DPC) framework for self-supervised representation learning on videos.  ...  -400 dataset with self-supervised learning, and then finetuning the representation on a downstream task, i.e. action recognition.  ...  DPC with self-supervised learning to further enhance the representation quality.  ... 
arXiv:1909.04656v3 fatcat:il453jejcncaxkylugylzq2va4

Self-supervised learning methods and applications in medical imaging analysis: A survey [article]

Saeed Shurrab, Rehab Duwairi
2021 arXiv   pre-print
Self-supervised learning is a recent training paradigm that enables learning robust representations without the need for human annotation which can be considered as an effective solution for the scarcity  ...  The article covers a set of the most recent self-supervised learning methods from the computer vision field as they are applicable to the medical imaging analysis and categorize them as predictive, generative  ...  Contrastive self-supervised learning Contrastive predictive coding Contrastive predictive coding (CPC), is a contrastive unsupervised representation learning proposed by Oord et al. [2018] that can fit  ... 
arXiv:2109.08685v2 fatcat:iu2zanqqrnaflawcxndb6xszgu

Data-Efficient Image Recognition with Contrastive Predictive Coding [article]

Olivier J. Hénaff, Aravind Srinivas, Jeffrey De Fauw, Ali Razavi, Carl Doersch, S. M. Ali Eslami, Aaron van den Oord
2020 arXiv   pre-print
We therefore revisit and improve Contrastive Predictive Coding, an unsupervised objective for learning such representations.  ...  Finally, this unsupervised representation substantially improves transfer learning to object detection on the PASCAL VOC dataset, surpassing fully supervised pre-trained ImageNet classifiers.  ...  Overview of the framework for semi-supervised learning with Contrastive Predictive Coding. Left: unsupervised pre-training with the spatial prediction task (See Section 2.1).  ... 
arXiv:1905.09272v3 fatcat:rlooggmxljaflnyjrws2ocgcuu

Visually Guided Self Supervised Learning of Speech Representations [article]

Abhinav Shukla, Konstantinos Vougioukas, Pingchuan Ma, Stavros Petridis, Maja Pantic
2020 arXiv   pre-print
Self supervised representation learning has recently attracted a lot of research interest for both the audio and visual modalities.  ...  This demonstrates the potential of visual supervision for learning audio representations as a novel way for self-supervised learning which has not been explored in the past.  ...  ACKNOWLEDGEMENTS We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan V GPU used for this research and Amazon Web Services for providing computational resources for  ... 
arXiv:2001.04316v2 fatcat:owtnx4mgw5bbtgjh7sd2wlaji4

Wav2vec-C: A Self-supervised Model for Speech Representation Learning [article]

Samik Sadhu, Di He, Che-Wei Huang, Sri Harish Mallidi, Minhua Wu, Ariya Rastrow, Andreas Stolcke, Jasha Droppo, Roland Maas
2021 arXiv   pre-print
This work is one of only a few studies of self-supervised learning on speech tasks with a large volume of real far-field labeled data.  ...  The proposed self-supervised model is trained on 10k hours of unlabeled data and subsequently used as the speech encoder in a RNN-T ASR model and fine-tuned with 1k hours of labeled data.  ...  The recently proposed wav2vec 2.0 [5] is one such self-supervised learning model that learns to predict masked out discrete speech encodings using a contextualized representation from a transformer model  ... 
arXiv:2103.08393v2 fatcat:2pypllr6ovgdddbyewpmrxekri

Revisiting Self-Supervised Visual Representation Learning

Alexander Kolesnikov, Xiaohua Zhai, Lucas Beyer
2019 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)  
We challenge a number of common practices in selfsupervised visual representation learning and observe that standard recipes for CNN design do not always translate to self-supervised representation learning  ...  Among a big body of recently proposed approaches for unsupervised learning of visual representations, a class of self-supervised techniques achieves superior performance on many challenging benchmarks.  ...  For instance, ResNet50 v1 excels when trained with the relative patch location self-supervision [7] , but produces suboptimal results when trained with the rotation self-supervision [11] .  ... 
doi:10.1109/cvpr.2019.00202 dblp:conf/cvpr/KolesnikovZB19 fatcat:hutezdahpndirit2ygw75wb6em

Vector Quantized Contrastive Predictive Coding for Template-based Music Generation [article]

Gaëtan Hadjeres, Léopold Crestel
2020 arXiv   pre-print
Our contribution is two-fold: First, we propose a self-supervised encoding technique, named Vector Quantized Contrastive Predictive Coding which allows to learn a meaningful assignment of the basic units  ...  variations is intimately linked to the problem of learning relevant high-level representations without supervision.  ...  Background Contrastive Predictive Coding Contrastive Predictive Coding (CPC) is a self-supervised representation learning technique relying on a contrastive objective on the latent representation itself  ... 
arXiv:2004.10120v1 fatcat:6ag5gzpwdzgxfh7q34i5svhyyi

Self-supervised Learning of Point Clouds via Orientation Estimation [article]

Omid Poursaeed, Tianxing Jiang, Han Qiao, Nayun Xu, Vladimir G. Kim
2020 arXiv   pre-print
In this paper, we leverage 3D self-supervision for learning downstream tasks on point clouds with fewer labels.  ...  Moreover, features learned by our model are complementary to other self-supervised methods and combining them leads to further performance improvement.  ...  Method Inspired by the success of self-supervised learning on images, we propose a self-supervised method to learn representations of point clouds. We consider the proxy task of rotation prediction.  ... 
arXiv:2008.00305v2 fatcat:y7kpxoju5baepj4cvpovny77zi

Time-series Change Point Detection with Self-Supervised Contrastive Predictive Coding [article]

Shohreh Deldari, Daniel V. Smith, Hao Xue, Flora D. Salim
2020 arXiv   pre-print
In this work, we propose TSCP2 a novel self-supervised technique for temporal change point detection, based on representation learning with Temporal Convolutional Network (TCN).  ...  Therefore, coming up with a self-supervised solution is a necessity these days.  ...  CONCLUSION We propose a novel self-supervised CPD method, − 2 for time series. − 2 learns an embedded representation predict a future interval of a times series from historical samples.  ... 
arXiv:2011.14097v2 fatcat:ihunzxfotzh6hplpsyuemwwow4

Temporal Consistency Objectives Regularize the Learning of Disentangled Representations [chapter]

Gabriele Valvano, Agisilaos Chartsias, Andrea Leo, Sotirios A. Tsaftaris
2019 Lecture Notes in Computer Science  
By introducing a self-supervised objective of predicting future cardiac phases we improve disentanglement.  ...  Here we build on recent innovations in style-content representations to learn anatomy, imaging characteristics (appearance) and temporal correlations.  ...  Conclusion We introduced a self-supervised objective for learning disentangled anatomymodality representations in cardiac imaging.  ... 
doi:10.1007/978-3-030-33391-1_2 fatcat:5g3zafgp2fdinp3f4k72mxzmo4

Sequential Adversarial Learning for Self-Supervised Deep Visual Odometry [article]

Shunkai Li, Fei Xue, Xin Wang, Zike Yan, Hongbin Zha
2019 arXiv   pre-print
Experiments on KITTI and Cityscapes datasets show that our method obtains more accurate depth with details preserved and predicted pose outperforms state-of-the-art self-supervised methods significantly  ...  We propose a self-supervised learning framework for visual odometry (VO) that incorporates correlation of consecutive frames and takes advantage of adversarial learning.  ...  In this paper, we propose to learn a compact representation (referred to as 'code') of the correlation between frames, and sequential information is accumulated by integrating codes via Long Short-Term  ... 
arXiv:1908.08704v1 fatcat:fz6aykzgsjft7lwjvsu5w7z3zi
« Previous Showing results 1 — 15 out of 55,223 results