Filters








205 Hits in 2.9 sec

Data Distillation: Towards Omni-Supervised Learning [article]

Ilija Radosavovic, Piotr Dollár, Ross Girshick, Georgia Gkioxari, Kaiming He
2017 arXiv   pre-print
We investigate omni-supervised learning, a special regime of semi-supervised learning in which the learner exploits all available labeled data plus internet-scale sources of unlabeled data.  ...  To exploit the omni-supervised setting, we propose data distillation, a method that ensembles predictions from multiple transformations of unlabeled data, using a single model, to automatically generate  ...  To test data distillation for omni-supervised learning, we evaluate it on the human keypoint detection task of the COCO dataset [24] .  ... 
arXiv:1712.04440v1 fatcat:f2vlejijqzgubg33u4zuanv65y

Data Distillation: Towards Omni-Supervised Learning

Ilija Radosavovic, Piotr Dollar, Ross Girshick, Georgia Gkioxari, Kaiming He
2018 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition  
We investigate omni-supervised learning, a special regime of semi-supervised learning in which the learner exploits all available labeled data plus internet-scale sources of unlabeled data.  ...  To exploit the omni-supervised setting, we propose data distillation, a method that ensembles predictions from multiple transformations of unlabeled data, using a single model, to automatically generate  ...  To test data distillation for omni-supervised learning, we evaluate it on the human keypoint detection task of the COCO dataset [24] .  ... 
doi:10.1109/cvpr.2018.00433 dblp:conf/cvpr/RadosavovicDGGH18 fatcat:h2flfwytfnan7ou5uprx3fs24a

Omni-supervised Facial Expression Recognition via Distilled Data [article]

Ping Liu, Yunchao Wei, Zibo Meng, Weihong Deng, Joey Tianyi Zhou, Yi Yang
2021 arXiv   pre-print
From a different perspective, we propose to perform omni-supervised learning to directly exploit reliable samples in a large amount of unlabeled data for network training.  ...  We experimentally verify that the new dataset created in such an omni-supervised manner can significantly improve the generalization ability of the learned FER model.  ...  “Data distillation: Towards omni-supervised learning,” in IEEE [22] Y. Li, J. Zeng, S. Shan, and X.  ... 
arXiv:2005.08551v5 fatcat:37lk5obe3vf7tigejcssl3gow4

Similarity-Preserving Knowledge Distillation [article]

Frederick Tung, Greg Mori
2019 arXiv   pre-print
For example, in neural network compression, a high-capacity teacher is distilled to train a compact student; in privileged learning, a teacher trained with privileged data is distilled to train a student  ...  without access to that data.  ...  As future work, we plan to explore similarity-preserving knowledge distillation in semisupervised and omni-supervised [29] learning settings.  ... 
arXiv:1907.09682v2 fatcat:up3tyyp74fb6boym3fxqxsa5ua

Attention-Guided Answer Distillation for Machine Reading Comprehension [article]

Minghao Hu, Yuxing Peng, Furu Wei, Zhen Huang, Dongsheng Li, Nan Yang, Ming Zhou
2018 arXiv   pre-print
We first demonstrate that vanilla knowledge distillation applied to answer span prediction is effective for reading comprehension systems.  ...  This paper tackles these problems by leveraging knowledge distillation, which aims to transfer knowledge from an ensemble model to a single model.  ...  Radosavovic et al. (2017) introduce data distillation that annotates large-scale unlabelled data for omni-supervised learning.  ... 
arXiv:1808.07644v4 fatcat:wcnhvuat5veelomvmj5rntwmre

Continual Reinforcement Learning deployed in Real-life using Policy Distillation and Sim2Real Transfer [article]

René Traoré, Hugo Caselles-Dupré, Timothée Lesort, Te Sun, Natalia Díaz-Rodríguez, David Filliat
2019 arXiv   pre-print
Our approach takes advantage of state representation learning and policy distillation.  ...  We provide preliminary work on applying Reinforcement Learning to such setting, on 2D navigation tasks for a 3 wheel omni-directional robot.  ...  The distillation consists in learning in a supervised fashion the action probability associated to an observation at a timestep t.  ... 
arXiv:1906.04452v1 fatcat:oaceeseydfa25hczcpob4oyypy

Omni-Training for Data-Efficient Deep Learning [article]

Yang Shu, Zhangjie Cao, Jinghan Gao, Ziyang Zhang, Jianmin Wang, Mingsheng Long
2022 arXiv   pre-print
This motivates the proposed Omni-Training framework towards data-efficient deep learning. Our first contribution is Omni-Net, a tri-flow architecture.  ...  Our second contribution is Omni-Loss, in which a self-distillation regularization is imposed to enable knowledge transfer across the training process.  ...  In this paper, we focus on representation learning algorithms towards data-efficiency, which aim to learn transferable representations from pretext data to reduce the data requirement of learning new tasks  ... 
arXiv:2110.07510v2 fatcat:phtuydvso5a4lfxgejua2i5eom

In Defense of the Triplet Loss Again: Learning Robust Person Re-Identification with Fast Approximated Triplet Loss and Label Distillation [article]

Ye Yuan, Wuyang Chen, Yang Yang, Zhangyang Wang
2019 arXiv   pre-print
A label distillation strategy is further designed to learn refined soft-labels in place of the potentially noisy labels, from only an identified subset of confident examples, through teacher-student networks  ...  The comparative losses (typically, triplet loss) are appealing choices for learning person re-identification (ReID) features.  ...  [28] further extended data distillation to omni-supervised learning by ensemble of predictions from multiple transformations of unlabeled data to generate new training annotations using a single network  ... 
arXiv:1912.07863v2 fatcat:z3z3wipixnhxvncthhn3s4ytxq

Small Sample Learning in Big Data Era [article]

Jun Shu, Zongben Xu, Deyu Meng
2018 arXiv   pre-print
This category mainly focuses on learning with insufficient samples, and can also be called small data learning in some literatures.  ...  The second category is called "experience learning", which usually co-exists with the large sample learning manner of conventional machine learning.  ...  Different from the above methods via model distillation, Radosavovic et al. (2018) investigated omni-supervised learning, a data distillation method, ensembling predictions from multiple transformations  ... 
arXiv:1808.04572v3 fatcat:lqqzzrmgfnfb3izctvdzgopuny

Look, Cast and Mold: Learning 3D Shape Manifold from Single-view Synthetic Data [article]

Qianyu Feng, Yawei Luo, Keyang Luo, Yi Yang
2021 arXiv   pre-print
Extensive experiments on several benchmarks demonstrate the effectiveness and robustness of the proposed method in learning the 3D shape manifold from synthetic data via a single-view.  ...  To equip deep models with this ability usually requires abundant 3D supervision which is hard to acquire.  ...  Towards this, numerous CAD objects are accessible during training as data samples from the source domain.  ... 
arXiv:2103.04789v2 fatcat:3hjvb4to5jffhnrc4i5ofu3zwi

Capturing Omni-Range Context for Omnidirectional Segmentation [article]

Kailun Yang, Jiaming Zhang, Simon Reiß, Xinxin Hu, Rainer Stiefelhagen
2021 arXiv   pre-print
In addition to the learned attention-based contextual priors that can stretch across 360-degree images, we upgrade model training by leveraging multi-source and omni-supervised learning, taking advantage  ...  of both: Densely labeled and unlabeled data originating from multiple datasets.  ...  We extend the design of [71] based on data distillation [46] and present a multisource omni-supervised learning regimen.  ... 
arXiv:2103.05687v1 fatcat:n3ej2ujenbhcnc7mtoufufcwai

Towards Omni-Supervised Face Alignment for Large Scale Unlabeled Videos [article]

Congcong Zhu, Hao Liu, Zhenhua Yu, Xuehong Sun
2019 arXiv   pre-print
In this paper, we propose a spatial-temporal relational reasoning networks (STRRN) approach to investigate the problem of omni-supervised face alignment in videos.  ...  Unlike existing fully supervised methods which rely on numerous annotations by hand, our learner exploits large scale unlabeled videos plus available labeled data to generate auxiliary plausible training  ...  To address this issue, we investigate into the omni-supervised learning towards face alignment inspired by (Radosavovic et al. 2018; Dong et al. 2018) .  ... 
arXiv:1912.07243v1 fatcat:3kjaq6dz7fh4dikzg5gw6nwvdu

Towards Omni-Supervised Face Alignment for Large Scale Unlabeled Videos

Congcong Zhu, Hao Liu*(corresponding author), Zhenhua Yu, Xuehong Sun
2020 PROCEEDINGS OF THE THIRTIETH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE AND THE TWENTY-EIGHTH INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE  
In this paper, we propose a spatial-temporal relational reasoning networks (STRRN) approach to investigate the problem of omni-supervised face alignment in videos.  ...  Unlike existing fully supervised methods which rely on numerous annotations by hand, our learner exploits large scale unlabeled videos plus available labeled data to generate auxiliary plausible training  ...  To address this issue, we investigate into the omni-supervised learning towards face alignment inspired by (Radosavovic et al. 2018; Dong et al. 2018) .  ... 
doi:10.1609/aaai.v34i07.7011 fatcat:kjwusgab3neajkz2pebculuhda

UFO^2: A Unified Framework towards Omni-supervised Object Detection [article]

Zhongzheng Ren, Zhiding Yu, Xiaodong Yang, Ming-Yu Liu, Alexander G. Schwing, Jan Kautz
2020 arXiv   pre-print
We also use UFO^2 to investigate budget-aware omni-supervised learning, i.e., various annotation policies are studied under a fixed annotation budget: we show that competitive performance needs no strong  ...  Specifically, UFO^2 incorporates strong supervision (e.g., boxes), various forms of partial supervision (e.g., class tags, points, and scribbles), and unlabeled data.  ...  Omni-supervised learning is particularly beneficial in practice.  ... 
arXiv:2010.10804v1 fatcat:such75nqvbbhtn7jrntzys77x4

Omni: Automated Ensemble with Unexpected Models against Adversarial Evasion Attack [article]

Rui Shu, Tianpei Xia, Laurie Williams, Tim Menzies
2021 arXiv   pre-print
Machine learning-based security detection models have become prevalent in modern malware and intrusion detection systems.  ...  Once the attackers can fool a classifier to think that a malicious input is actually benign, they can render a machine learning-based malware or intrusion detection system ineffective.  ...  to detect Android malware [34] ; designing supervised learning algorithm to classify HTTP logs [49] ; designing machine learning models to detect ransomware [57] ; and detecting malicious PowerShell  ... 
arXiv:2011.12720v2 fatcat:g755b3q2lnem7irwin5o2kh67i
« Previous Showing results 1 — 15 out of 205 results