Filters








7,033 Hits in 4.2 sec

Should We Be Pre-training? An Argument for End-task Aware Training as an Alternative [article]

Lucio M. Dery, Paul Michel, Ameet Talwalkar, Graham Neubig
2022 arXiv   pre-print
We next introduce an online meta-learning algorithm that learns a set of multi-task weights to better balance among our multiple auxiliary objectives, achieving further improvements on end-task performance  ...  widely-used task-agnostic continued pre-training paradigm of Gururangan et al. (2020).  ...  end-task agnostic continued pre-training, we suggest introducing the end-task objective into the training process via multi-task learning (Caruana, 1997; Ruder, 2017) .  ... 
arXiv:2109.07437v2 fatcat:ehuvvggh2ba7lfs6jeg6w3ss3u

Task-Agnostic Continual Reinforcement Learning: In Praise of a Simple Baseline [article]

Massimo Caccia, Jonas Mueller, Taesup Kim, Laurent Charlin, Rasool Fakoor
2022 arXiv   pre-print
Our hypotheses are empirically tested in continuous control tasks via a large-scale study of the popular multi-task and continual learning benchmark Meta-World.  ...  We study task-agnostic continual reinforcement learning (TACRL) in which standard RL challenges are compounded with partial observability stemming from task agnosticism, as well as additional difficulties  ...  task-agnostic continual reinforcement learning (TACRL), and contrast it against multi-task RL as well as task-aware settings.  ... 
arXiv:2205.14495v1 fatcat:ubdhbioejvfbdgu6lot5ii4eka

Self-Attention Meta-Learner for Continual Learning [article]

Ghada Sokar, Decebal Constantin Mocanu, Mykola Pechenizkiy
2021 arXiv   pre-print
In this paper, we propose a new method, named Self-Attention Meta-Learner (SAM), which learns a prior knowledge for continual learning that permits learning a sequence of tasks, while avoiding catastrophic  ...  Continual learning aims to provide intelligent agents capable of learning multiple tasks sequentially with neural networks.  ...  Therefore, we propose to learn this representation via meta-learning to permit generalization to out-of-distribution tasks.  ... 
arXiv:2101.12136v1 fatcat:2lpbi5gxjradvifbbdarx4ifri

Meta-Learning of Neural Architectures for Few-Shot Learning

Thomas Elsken, Benedikt Staffler, Jan Hendrik Metzen, Frank Hutter
2020 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)  
Moreover, METANAS is agnostic in that it can be used with arbitrary model-agnostic meta-learning algorithms and arbitrary gradient-based NAS methods.  ...  During meta-testing, architectures can be adapted to a novel task with a few steps of the task optimizer, that is: task adaptation becomes computationally cheap and requires only little data per task.  ...  via the learned system on new tasks.  ... 
doi:10.1109/cvpr42600.2020.01238 dblp:conf/cvpr/ElskenSMH20 fatcat:iwdphy5klre3vdwgi73ra7itdy

Meta-Learning of Neural Architectures for Few-Shot Learning [article]

Thomas Elsken, Benedikt Staffler, Jan Hendrik Metzen, Frank Hutter
2021 arXiv   pre-print
Moreover, MetaNAS is agnostic in that it can be used with arbitrary model-agnostic meta-learning algorithms and arbitrary gradient-based NAS methods.  ...  During meta-testing, architectures can be adapted to a novel task with a few steps of the task optimizer, that is: task adaptation becomes computationally cheap and requires only little data per task.  ...  via the learned system on new tasks.  ... 
arXiv:1911.11090v3 fatcat:q42t62fh6rfchlyxllilxrqha4

Learning to Learn: Meta-Critic Networks for Sample Efficient Learning [article]

Flood Sung, Li Zhang, Tao Xiang, Timothy Hospedales, Yongxin Yang
2017 arXiv   pre-print
The key idea is to learn a meta-critic: an action-value function neural network that learns to criticise any actor trying to solve any specified task.  ...  We propose a novel and flexible approach to meta-learning for learning-to-learn from only a few examples.  ...  that the task-agnostic knowledge is encoded in the weights of the meta-network.  ... 
arXiv:1706.09529v1 fatcat:ryxe5uebkjagvabw42epozukfi

iTAML: An Incremental Task-Agnostic Meta-learning Approach

Jathushan Rajasegaran, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Mubarak Shah
2020 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)  
This is ensured by a new meta-update rule which avoids catastrophic forgetting. In comparison to previous metalearning techniques, our approach is task-agnostic.  ...  Humans can continuously learn new knowledge as their experience grows. In contrast, previous learning in deep neural networks can quickly fade out when they are trained on a new task.  ...  Incremental Task Agnostic Meta-learning We progressively learn a total of T tasks, with U number of classes per task.  ... 
doi:10.1109/cvpr42600.2020.01360 dblp:conf/cvpr/RajasegaranKHKS20 fatcat:3n6frqycnfdfve66owocqrd3pe

Model-Agnostic Learning to Meta-Learn [article]

Arnout Devos, Yatin Dandi
2021 arXiv   pre-print
Experiments on more complex image classification, continual regression, and reinforcement learning tasks demonstrate that learning to meta-learn generally improves task-specific adaptation.  ...  We investigate how learning with different task distributions can first improve adaptability by meta-finetuning on related tasks before improving goal task generalization with finetuning.  ...  A Model-Agnostic Learning to Meta-Learn Algorithm We propose a method that can learn the parameters of any model via learning to meta-learn in such a way as to prepare that model to first quickly change  ... 
arXiv:2012.02684v2 fatcat:xbrjsoxgdva3rba5zlsnyqjiou

StyleMeUp: Towards Style-Agnostic Sketch-Based Image Retrieval [article]

Aneeshan Sain, Ayan Kumar Bhunia, Yongxin Yang, Tao Xiang, Yi-Zhe Song
2021 arXiv   pre-print
With this meta-learning framework, our model can not only disentangle the cross-modal shared semantic content for SBIR, but can adapt the disentanglement to any unseen user style as well, making the SBIR  ...  model truly style-agnostic.  ...  Meta-Learning: Meta-learning aims to acquire transferable knowledge from different sample training-tasks to help adapt to unseen tasks with only a few training samples [18] .  ... 
arXiv:2103.15706v2 fatcat:ukbeu2bpujb53j3pzd5zbdldai

iTAML: An Incremental Task-Agnostic Meta-learning Approach [article]

Jathushan Rajasegaran, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Mubarak Shah
2020 arXiv   pre-print
This is ensured by a new meta-update rule which avoids catastrophic forgetting. In comparison to previous meta-learning techniques, our approach is task-agnostic.  ...  Humans can continuously learn new knowledge as their experience grows. In contrast, previous learning in deep neural networks can quickly fade out when they are trained on a new task.  ...  Incremental Task Agnostic Meta-learning We progressively learn a total of T tasks, with U number of classes per task.  ... 
arXiv:2003.11652v1 fatcat:q3uierncg5fn7fy66rzkgpef5q

Multimodal Model-Agnostic Meta-Learning via Task-Aware Modulation [article]

Risto Vuorio, Shao-Hua Sun, Hexiang Hu, Joseph J. Lim
2019 arXiv   pre-print
Model-agnostic meta-learners aim to acquire meta-learned parameters from similar tasks to adapt to novel tasks from the same distribution with few gradient updates.  ...  The results not only demonstrate the effectiveness of our model in modulating the meta-learned prior in response to the characteristics of tasks but also show that training on a multimodal distribution  ...  We consider the following model-agnostic meta-learning baselines: • MAML [8] represents the family of model-agnostic meta-learners.  ... 
arXiv:1910.13616v1 fatcat:fq5lgtkpwrgjxatzca4puvpf6y

Meta-learnt priors slow down catastrophic forgetting in neural networks [article]

Giacomo Spigler
2020 arXiv   pre-print
Finally, we present SeqFOMAML, a meta-learning algorithm that implements these principles, and we evaluate it on sequential learning problems composed by Omniglot and MiniImageNet classification tasks.  ...  Here we show that catastrophic forgetting can be mitigated in a meta-learning context, by exposing a neural network to multiple tasks in a sequential manner during training.  ...  Meta-continual learning.  ... 
arXiv:1909.04170v2 fatcat:lf7djggjwbhafcjkl6cglrde6e

Data Efficient Direct Speech-to-Text Translation with Modality Agnostic Meta-Learning [article]

Sathish Indurthi, Houjeung Han, Nikhil Kumar Lakumarapu, Beomseok Lee, Insoo Chung, Sangha Kim, Chanwoo Kim
2020 arXiv   pre-print
In this work, we adopt a meta-learning algorithm to train a modality agnostic multi-task model that transfers knowledge from source tasks=ASR+MT to target task=ST where ST task severely lacks data.  ...  We evaluate the proposed meta-learning approach for ST tasks on English-German (En-De) and English-French (En-Fr) language pairs from the Multilingual Speech Translation Corpus (MuST-C).  ...  Transfer Learning: Here, we compare the performance of Baseline 2, Baseline 3, and the proposed modality agnostic meta-learning model for ST task.  ... 
arXiv:1911.04283v2 fatcat:frhk6t6h2jf3fhttkmikooomcy

Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks [article]

Chelsea Finn, Pieter Abbeel, Sergey Levine
2017 arXiv   pre-print
The goal of meta-learning is to train a model on a variety of learning tasks, such that it can solve new learning tasks using only a small number of training samples.  ...  We propose an algorithm for meta-learning that is model-agnostic, in the sense that it is compatible with any model trained with gradient descent and applicable to a variety of different learning problems  ...  Diagram of our model-agnostic meta-learning algorithm (MAML), which optimizes for a representation θ that can quickly adapt to new tasks.  ... 
arXiv:1703.03400v3 fatcat:c2f3ayn6kretpb6vga7hrziwoi

Continual learning under domain transfer with sparse synaptic bursting [article]

Shawn L. Beaulieu, Jeff Clune, Nick Cheney
2022 arXiv   pre-print
We find that our method learns continually under domain transfer with sparse bursts of activity in weights that are recycled across tasks, rather than by maintaining task-specific modules.  ...  This has not yet enabled continual learning over long sequences of previously unseen data without corrupting existing knowledge: a problem known as catastrophic forgetting.  ...  This work is supported in part by the DARPA Lifelong Learning Machines award HR0011-18-2-0018. We'd like to thank Josh C.  ... 
arXiv:2108.12056v7 fatcat:wnwgcsxi6bc5bkk2ss2jwvdv74
« Previous Showing results 1 — 15 out of 7,033 results