Filters








136,965 Hits in 4.4 sec

Meta-Learning to Improve Pre-Training [article]

Aniruddh Raghu, Jonathan Lorraine, Simon Kornblith, Matthew McDermott, David Duvenaud
2021 arXiv   pre-print
Pre-training (PT) followed by fine-tuning (FT) is an effective method for training neural networks, and has led to significant performance improvements in many domains.  ...  In this work, we propose an efficient, gradient-based algorithm to meta-learn PT hyperparameters.  ...  The authors thank the members of the Clinical and Applied Machine Learning group at MIT and Paul Vicol for helpful feedback.  ... 
arXiv:2111.01754v1 fatcat:flzc3rdvnbc5hfxqukhzh7nhqq

Omni-Training for Data-Efficient Deep Learning [article]

Yang Shu, Zhangjie Cao, Jinghan Gao, Ziyang Zhang, Jianmin Wang, Mingsheng Long
2022 arXiv   pre-print
Transferability has become the key to enable data-efficient deep learning, however, existing pre-training methods focus only on domain transferability while meta-training methods only on task transferability  ...  Omni-Training is a general framework that accommodates many existing pre-training and meta-training algorithms.  ...  Yang Shu and Zhangjie Cao contributed equally to this work. Correspondences shall be addressed to Mingsheng Long.  ... 
arXiv:2110.07510v2 fatcat:phtuydvso5a4lfxgejua2i5eom

A Brief Summary of Interactions Between Meta-Learning and Self-Supervised Learning [article]

Huimin Peng
2021 arXiv   pre-print
Meta-learning can be applied to improve model generalization capability and to construct general AI algorithms.  ...  Meta-learning aims to adapt trained deep models to solve diverse tasks and to develop general AI algorithms.  ...  Pre-training uses large deep models to learn higher-level representations. In meta-learning, pre-trained deep models are adapted directly to solve unseen tasks efficiently.  ... 
arXiv:2103.00845v2 fatcat:soq6tfl56vgshebtnot57e4qwe

Meta-Learning for Fast Cross-Lingual Adaptation in Dependency Parsing [article]

Anna Langedijk, Verna Dankers, Phillip Lippe, Sander Bos, Bryan Cardenas Guevara, Helen Yannakoudakis, Ekaterina Shutova
2022 arXiv   pre-print
We find that meta-learning with pre-training can significantly improve upon the performance of language transfer and standard supervised learning baselines for a variety of unseen, typologically diverse  ...  Meta-learning, or learning to learn, is a technique that can help to overcome resource scarcity in cross-lingual NLP problems, by enabling fast adaptation to new tasks.  ...  This suggests that pre-training provides a better starting point for meta-learning than plain mBERT.  ... 
arXiv:2104.04736v3 fatcat:yptu2e7lkzhu3evfn4yzgvc67y

An Empirical Study on the Relation between Meta-cognitive Strategies and Listening Autonomous Learning Ability

Hui Guo
2012 Theory and Practice in Language Studies  
Thus, we think that meta-cognition is the most crucial to further improve learners' listening autonomous learning ability.  ...  Based on the essence of meta-cognition theory and the characteristics of college English teaching, the cultivation of students' meta-cognition is favorable for the improvement of their ability to learn  ...  The results indicates that the training of meta-cognition strategies can not only help to improve students English listening, but also inspire student"s motive and improve their ability to learn independently  ... 
doi:10.4304/tpls.2.11.2446-2451 fatcat:s5rmnpatebgvxhapxzrgvwvgey

A Closer Look at Few-Shot Video Classification: A New Baseline and Benchmark [article]

Zhenxi Zhu, Limin Wang, Sheng Guo, Gangshan Wu
2021 arXiv   pre-print
While significant progress has been made, these methods fail to focus on learning effective representations, and heavily rely on the ImageNet pre-training, which might be unreasonable for the few-shot  ...  Finally, we present a new benchmark with more base data to facilitate future few-shot video classification without pre-training.  ...  Our classifier-based methods gain a large performance improvement compared to meta-learning methods.  ... 
arXiv:2110.12358v1 fatcat:7fwo6wbv6bbrjjwzz4wf2ssb7y

Should We Be Pre-training? An Argument for End-task Aware Training as an Alternative [article]

Lucio M. Dery, Paul Michel, Ameet Talwalkar, Graham Neubig
2022 arXiv   pre-print
We next introduce an online meta-learning algorithm that learns a set of multi-task weights to better balance among our multiple auxiliary objectives, achieving further improvements on end-task performance  ...  In most settings of practical concern, machine learning practitioners know in advance what end-task they wish to boost with auxiliary tasks.  ...  Our META-learning end-Task AwaRe TrAiniNg (META-TARTAN) allows us to robustly modulate between multiple objectives and further improves performance over MT-TARTAN . • A naive implementation of META-TARTAN  ... 
arXiv:2109.07437v2 fatcat:ehuvvggh2ba7lfs6jeg6w3ss3u

Revisiting Meta-Learning as Supervised Learning [article]

Wei-Lun Chao, Han-Jia Ye, De-Chuan Zhan, Mark Campbell, Kilian Q. Weinberger
2020 arXiv   pre-print
This view not only unifies meta-learning into an intuitive and practical framework but also allows us to transfer insights from supervised learning directly to improve meta-learning.  ...  We provide an intuitive analogy of these methods in the context of meta-learning and show that they give rise to significant improvements in model performance on few-shot learning.  ...  Multi-objective learning does not increase the meta-training accuracy much but improves the meta-test accuracy, verifying its ability to improve generalization.  ... 
arXiv:2002.00573v1 fatcat:f5fcejzwmrgbjp4pn64fvqp43a

Test-time Adaptation for Real Image Denoising via Meta-transfer Learning [article]

Agus Gunawan, Muhammad Adi Nugroho, Se Jin Park
2022 arXiv   pre-print
The learning strategy is two stages where the first stage pre-train the network using meta-auxiliary learning to get better meta-initialization.  ...  Meanwhile, we use meta-learning for fine-tuning (meta-transfer learning) the network as the second stage of our training to enable test-time adaptation on real noisy images.  ...  The first stage pre-train the network using a meta-auxiliary learning algorithm to get better meta-initialization.  ... 
arXiv:2207.02066v1 fatcat:a4cf3uplznbo5decpaxante4zm

MetaVA: Curriculum Meta-learning and Pre-fine-tuning of Deep Neural Networks for Detecting Ventricular Arrhythmias based on ECGs [article]

Wenrui Zhang, Shijia Geng, Zhaoji Fu, Linlin Zheng, Chenyang Jiang, Shenda Hong
2022 arXiv   pre-print
CL is supposed to further improve MAML by meta-learning from easy to difficult tasks. For the fine-tuning stage, we propose improved pre-fine-tuning to solve individual-level diversity.  ...  For the pre-training stage, we propose a novel model agnostic meta-learning (MAML) with curriculum learning (CL) method to solve group-level diversity.  ...  First, we combine a meta-learning method, model agnostic meta-learning (MAML), and a curriculum learning (CL) strategy when pre-training the model.  ... 
arXiv:2202.12450v2 fatcat:c6bobcfcorgyrpjruwkqfiux5a

Representation based and Attention augmented Meta learning [article]

Yunxiao Qin, Chenxu Zhao, Zezheng Wang, Junliang Xing, Jun Wan, Zhen Lei
2018 arXiv   pre-print
Recently, Meta learning algorithm has been confirmed as a promising way to improve the ability of learning from few images for computer vision.  ...  The method AML aims to improve the Meta learner's attention ability by explicitly embedding an attention model into its network.  ...  In the Meta training stage, for the Meta learner not forgetting the learned knowledge, we fix the pre-trained representation module totally, and the Meta learner only needs to learn how to solve the few  ... 
arXiv:1811.07545v3 fatcat:ykm2jzxc45a2za2in2raeoxi34

Self-Supervised Video Representation Learning with Meta-Contrastive Network [article]

Yuanze Lin, Xun Guo, Yan Lu
2021 arXiv   pre-print
Self-supervised learning has been successfully applied to pre-train video representations, which aims at efficient adaptation from pre-training domain to downstream tasks.  ...  Our method contains two training stages based on model-agnostic meta learning (MAML), each of which consists of a contrastive branch and a meta branch.  ...  How to improve the generalization of contrastive selfsupervised learning and make the learned parameters easily adapt from pre-training domain to fine-tuning domain for various new tasks is still challenging  ... 
arXiv:2108.08426v2 fatcat:bi532velizgg7pr7hzxbllgahy

Self-Paced Contrastive Learning for Semi-supervised Medical Image Segmentation with Meta-labels [article]

Jizong Peng, Ping Wang, Chrisitian Desrosiers, Marco Pedersoli
2021 arXiv   pre-print
We use the meta-labels for pre-training the image encoder as well as to regularize a semi-supervised training, in which a reduced set of annotated data is used for training.  ...  In this work, we propose to adapt contrastive learning to work with meta-label annotations, for improving the model's performance in medical image segmentation even when no additional unlabeled data is  ...  It can be seen that performance is inferior to Contrastive pre-training on meta-data, however the improvement is still quite relevant and, in most cases, superior to unsupervised pre-training.  ... 
arXiv:2107.13741v2 fatcat:z3pozdnpozg73j7vj6hb6pc5la

Towards Enabling Meta-Learning from Target Models [article]

Su Lu, Han-Jia Ye, Le Gan, De-Chuan Zhan
2021 arXiv   pre-print
Meta-learning can extract an inductive bias from previous learning experience and assist the training of new tasks.  ...  We find that with a small ratio of tasks armed with target models, classic meta-learning algorithms can be improved a lot without consuming many resources.  ...  In this experiment, we assume that target models for all meta-training tasks are  ... 
arXiv:2104.03736v5 fatcat:qzurcakghjb7hkwb73xjuftdlm

Multimodal Few-Shot Object Detection with Meta-Learning Based Cross-Modal Prompting [article]

Guangxing Han, Jiawei Ma, Shiyuan Huang, Long Chen, Rama Chellappa, Shih-Fu Chang
2022 arXiv   pre-print
We first show that meta-learning and prompt-based learning, the most commonly-used methods for few-shot learning and zero-shot transferring from pre-trained vision-language models to downstream tasks,  ...  Specifically, to better exploit the pre-trained vision-language models, the meta-learning based cross-modal prompting is proposed to generate soft prompts and further used to extract the semantic prototype  ...  Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation therein.  ... 
arXiv:2204.07841v1 fatcat:uxrlum2clzhhjbh67iugwcvwnu
« Previous Showing results 1 — 15 out of 136,965 results