A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Filters
Exploring the Limits of Weakly Supervised Pretraining
[article]
2018
arXiv
pre-print
Preserved Fulltext
State-of-the-art visual perception models for a wide range of tasks rely on supervised pretraining. ImageNet classification is the de facto pretraining task for these models. ...
Even so, relatively little is known about the behavior of pretraining with datasets that are multiple orders of magnitude larger. ...
Discussion We have attempted to explore the limits of supervised pretraining. ...
arXiv:1805.00932v1
fatcat:3qro6b6sindwtnvfzjbna4lufm
Exploring the Limits of Weakly Supervised Pretraining
[chapter]
2018
Lecture Notes in Computer Science
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
State-of-the-art visual perception models for a wide range of tasks rely on supervised pretraining. ImageNet classification is the de facto pretraining task for these models. ...
Even so, relatively little is known about the behavior of pretraining with datasets that are multiple orders of magnitude larger. ...
Exploring the Limits of Weakly Supervised Pretraining ...
doi:10.1007/978-3-030-01216-8_12
fatcat:vldy33o7tfchzix7gvqajujala
Semi-weakly Supervised Contrastive Representation Learning for Retinal Fundus Images
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
We explore the value of weak labels in learning transferable representations for medical images. ...
We consider weak labels in the form of pseudo-labels and propose a semi-weakly supervised contrastive learning (SWCL) framework for representation learning using semi-weakly annotated images. ...
prompts us to explore the use of weak labels to inject information about lesions into image patches. ...
arXiv:2108.02122v1
fatcat:mb6tdf6sgjenpkrybq6s4rcvmi
Open-Vocabulary Object Detection Using Captions
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Weakly supervised and zero-shot learning techniques have been explored to scale object detectors to more categories with less supervision, but they have not been as successful and widely adopted as supervised ...
In this paper, we put forth a novel formulation of the object detection problem, namely open-vocabulary object detection, which is more general, more practical, and more effective than weakly supervised ...
We are inspired by the rich literature of weakly supervised visual grounding methods [42, 9, 5, 1] to design our imagecaption pretraining technique. ...
arXiv:2011.10678v2
fatcat:ven4oegqnrdilb4reguistgxnm
Self-supervised Pretraining of Visual Features in the Wild
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Other Versions
However, the premise of self-supervised learning is that it can learn from any random image and from any unbounded dataset. ...
Our final SElf-supERvised (SEER) model, a RegNetY with 1.3B parameters trained on 1B random images with 512 GPUs achieves 84.2% top-1 accuracy, surpassing the best self-supervised pretrained model by 1% ...
While the benefit of pretraining has been demonstrated in computer vision, it has been in the limited scope of curated datasets originally collected for supervised or weakly supervised learning. ...
arXiv:2103.01988v2
fatcat:u4qjr2aq45ahxiokncoi75jif4
Weakly Supervised Context Encoder using DICOM metadata in Ultrasound Imaging
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
In this work, we leverage DICOM metadata from ultrasound images to help learn representations of the ultrasound image. ...
Modern deep learning algorithms geared towards clinical adaption rely on a significant amount of high fidelity labeled data. ...
The authors are solely responsible for the content, and the work does not represent the official views of the National Institutes of Health. ...
arXiv:2003.09070v1
fatcat:x7orhc5hwfb4njhn464zmbcmqm
Pretrained Encoders are All You Need
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Our results show that pretrained representations are at par with state-of-the-art self-supervised methods trained on domain-specific data. ...
We also explore fine-tuning pretrained representations with self-supervised techniques, i.e., contrastive predictive coding, spatio-temporal contrastive learning, and augmentations. ...
We also investigate self-supervised finetuning of pretrained representations using state-of-the-art (SotA) self-supervised methods. ...
arXiv:2106.05139v1
fatcat:qkfqyxpkindlxollaxucut56uu
Watch, Listen and Tell: Multi-modal Weakly Supervised Dense Event Captioning
[article]
2019
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Specifically, we focus on the problem of weakly-supervised dense event captioning in videos and show that audio on its own can nearly rival performance of a state-of-the-art visual model and, combined ...
However, much of the research has been limited to approaches that either do not take audio corresponding to video into account at all, or those that model the audio-visual correlations in service of sound ...
Acknowledgments: This work was funded in part by the Vector Institute for AI, Canada CIFAR AI Chair, NSERC Canada Research Chair (CRC) and an NSERC Discovery and Discovery Accelerator Supplement Grants ...
arXiv:1909.09944v2
fatcat:paj6fq6mwvdmfk73ubfz55jg4a
Depth CNNs for RGB-D scene recognition: learning from scratch better than transferring from RGB-CNNs
[article]
2018
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
In contrast, we focus on the bottom layers, and propose an alternative strategy to learn depth features combining local weakly supervised training from patches followed by global fine tuning with images ...
In contrast, current RGB-D scene data is much more limited, so often leverages RGB large datasets, by transferring pretrained RGB CNN models and fine-tuning with the target RGB-D dataset. ...
Weakly supervised pretrained CNN It is difficult to learn deep CNN from scratch with depth images, due to the lack of enough training data. ...
arXiv:1801.06797v1
fatcat:bde4sdx7arehhhqvada57rifnm
SimVLM: Simple Visual Language Model Pretraining with Weak Supervision
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Other Versions
However, the requirement for expensive annotations including clean image captions and regional labels limits the scalability of existing approaches, and complicates the pretraining procedure with the introduction ...
Without utilizing extra data or task-specific customization, the resulting model significantly outperforms previous pretraining methods and achieves new state-of-the-art results on a wide range of discriminative ...
Srinivasan, Samira Daruki, Nan Du and Aashi Jain for help with data preparation, Chao Jia, Zhen Li, Jonathan Shen, Colin Raffel and Sharan Narang for assistance on experimental settings, and others in the ...
arXiv:2108.10904v3
fatcat:glozbeeytvdyvcgl7ersyz4i34
Billion-Scale Pretraining with Vision Transformers for Multi-Task Visual Representations
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
In this work, we describe how we (1) generate a dataset with over a billion images via large weakly-supervised pretraining to improve the performance of these visual representations, and (2) leverage Transformers ...
To support this backbone model, we detail a systematic approach to deriving weakly-supervised image annotations from heterogenous text signals, demonstrating the benefits of clustering techniques to handle ...
The authors would like to thank Eric Tzeng, Raymond Shiau, Kofi Boakye, Vahid Kazemi, and Chuck Rosenberg for valuable discussions regarding the paper, and the anonymous reviewers and ACs for their helpful ...
arXiv:2108.05887v1
fatcat:gm5lzf4pkrg3zez7unuq7epp3a
Learning Effective RGB-D Representations for Scene Recognition
2019
IEEE Transactions on Image Processing
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Note that this fulltext copy is not of the "primary" version of this work. The version it corresponds to is:
2018 pre-print arXiv:1809.06269v1 fatcat:yuhyrmc7kncffdvroxrhj66zeq
The first limitation is the lack of depth data for training deep learning models. ...
using weak supervision via patches. ...
A smaller architecture and a weakly supervised pretraining strategy for the bottom layers enables us to overcome the problem of very limited depth data. ...
doi:10.1109/tip.2018.2872629
pmid:30281448
fatcat:lnhv5g46s5dpngmstriivjncgy
Low-Resource Machine Translation Training Curriculum Fit for Low-Resource Languages
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Note that this fulltext copy is not of the "primary" version of this work. The version it corresponds to is:
2021 pre-print arXiv:2103.13272v1 fatcat:pgqktigslndrfkhobupgo37h2a
to Kazakh, showcasing the potential of weakly-supervised NMT for the low-resource languages. ...
When trained on supervised data, our training curriculum achieves a new state-of-the-art result on the Somali dataset (BLEU of 29.3 for Somali to English). ...
After pretraining the LM, we train a NMT model in an unsupervised or weakly-supervised (using comparable data) manner. ...
arXiv:2103.13272v2
fatcat:l7ttvdozivalpdjj4jnvdduaju
Weakly Supervised 3D Object Detection from Point Clouds
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
The source code and pretrained models are publicly available at https://github.com/Zengyi-Qin/Weakly-Supervised-3D-Object-Detection. ...
Weakly supervised learning is a promising approach to reducing the annotation requirement, but existing weakly supervised object detectors are mostly for 2D detection rather than 3D. ...
image recognition network pretrained on existing Figure 1 : Overview of the proposed weakly supervised 3D object detection framework. ...
arXiv:2007.13970v1
fatcat:2n6irzxxh5cj7mjckign3knlsq
Pretrained Encyclopedia: Weakly Supervised Knowledge-Pretrained Language Model
[article]
2019
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Recent breakthroughs of pretrained language models have shown the effectiveness of self-supervised learning for a wide range of natural language processing (NLP) tasks. ...
Moreover, we propose a simple yet effective weakly supervised pretraining objective, which explicitly forces the model to incorporate knowledge about real-world entities. ...
CONCLUSION We introduce a weakly supervised method to encourage pretrained language models to learn entitylevel knowledge. ...
arXiv:1912.09637v1
fatcat:jgh27a23fnfj5bcxjlbktxpq3m
« Previous
Showing results 1 — 15 out of 2,875 results