Filters








607 Hits in 6.5 sec

Piggyback: Adapting a Single Network to Multiple Tasks by Learning to Mask Weights [article]

Arun Mallya, Dillon Davis, Svetlana Lazebnik
2018 arXiv   pre-print
This work presents a method for adapting a single, fixed deep neural network to multiple tasks without affecting performance on already learned tasks.  ...  By building upon ideas from network quantization and pruning, we learn binary masks that piggyback on an existing network, or are applied to unmodified weights of that network to provide good performance  ...  Further, an unlimited number of tasks can piggyback onto a backbone network by learning a new mask.  ... 
arXiv:1801.06519v2 fatcat:fszexmtcwrcefgfn2maqjqfuxu

Piggyback: Adapting a Single Network to Multiple Tasks by Learning to Mask Weights [chapter]

Arun Mallya, Dillon Davis, Svetlana Lazebnik
2018 Lecture Notes in Computer Science  
This work presents a method for adapting a single, fixed deep neural network to multiple tasks without affecting performance on already learned tasks.  ...  Fig. 1 : Overview of our method for learning piggyback masks for fixed backbone networks.  ...  Further, an unlimited number of tasks can piggyback onto a backbone network by learning a new mask.  ... 
doi:10.1007/978-3-030-01225-0_5 fatcat:inx2zano6vhsjdp6yj5nw2rona

Adding New Tasks to a Single Network with Weight Transformations using Binary Masks [article]

Massimiliano Mancini, Elisa Ricci, Barbara Caputo, Samuel Rota Bulò
2018 arXiv   pre-print
Recent work has shown that masking the internal weights of a given original conv-net through learned binary variables is a promising strategy.  ...  Given a deep model trained on a specific, given task, it would be highly desirable to be able to adapt incrementally to new tasks, preserving scalability as the number of new tasks increases, while at  ...  Second, it should be avoided adding multiple parameters to the model for each new task learned, as it would lead to poor scalability of the framework.  ... 
arXiv:1805.11119v2 fatcat:66mmtxq5yjhh3akasib4o2cjfu

KSM: Fast Multiple Task Adaption via Kernel-wise Soft Mask Learning [article]

Li Yang, Zhezhi He, Junshan Zhang, Deliang Fan
2020 arXiv   pre-print
Recently, the fast mask-based learning method (e.g. piggyback ) is proposed to address these issues by learning only a binary element-wise mask in a fast manner, while keeping the backbone model fixed.  ...  Deep Neural Networks (DNN) could forget the knowledge about earlier tasks when learning new tasks, and this is known as catastrophic forgetting.  ...  Give a task t, we aim to learn a task-specific soft mask M t , by refactoring the fixed backbone weight to favor the current task.  ... 
arXiv:2009.05668v1 fatcat:jem2uw7zcjdlzlirqskfw3aaui

Boosting binary masks for multi-domain learning through affine transformations

Massimiliano Mancini, Elisa Ricci, Barbara Caputo, Samuel Rota Bulò
2020 Machine Vision and Applications  
Given a pretrained architecture and a set of visual domains received sequentially, the goal of multi-domain learning is to produce a single model performing a task in all the domains together.  ...  In this work, we provide a general formulation of binary mask based models for multi-domain learning by affine transformations of the original network parameters.  ...  Acknowledgements We acknowledge financial support from ERC grant 637076 -RoboExNovo and project DIGIMAP, grant 860375, funded by the Austrian Research Promotion Agency (FFG).  ... 
doi:10.1007/s00138-020-01090-5 fatcat:xky4x527erhlpn3rv4ma3qzsee

DA^3:Dynamic Additive Attention Adaption for Memory-EfficientOn-Device Multi-Domain Learning [article]

Li Yang, Adnan Siraj Rakin, Deliang Fan
2021 arXiv   pre-print
Nowadays, one practical limitation of deep neural network (DNN) is its high degree of specialization to a single task or domain (e.g., one visual domain).  ...  training time by 2x, in comparison to the baseline methods (e.g., standard fine-tuning, Parallel and Series Res. adaptor, and Piggyback).  ...  Introduction Nowadays, one practical limitation of deep neural network (DNN) is its high degree of specialization to a single task or domain (e.g., one visual domain).  ... 
arXiv:2012.01362v3 fatcat:y72glgirnzgkvnuaxf26ewuxhe

Scalable and Order-robust Continual Learning with Additive Parameter Decomposition [article]

Jaehong Yoon, Saehoon Kim, Eunho Yang, Sung Ju Hwang
2020 arXiv   pre-print
First, a continual learning model should effectively handle catastrophic forgetting and be efficient to train even with a large number of tasks.  ...  We validate our network with APD, APD-Net, on multiple benchmark datasets against state-of-the-art continual learning methods, which it largely outperforms in accuracy, scalability, and order-robustness  ...  We use VGG-16 as the base network, and compare against an additional baseline, Piggyback (Mallya et al., 2018) , which handles a newly arrived task by learning a task-specific binary mask on a network  ... 
arXiv:1902.09432v3 fatcat:wf43nmbhmjcmbkgeqatbek2dpe

Ternary Feature Masks: zero-forgetting for task-incremental learning [article]

Marc Masana, Tinne Tuytelaars, Joost van de Weijer
2021 arXiv   pre-print
By using ternary masks we can upgrade a model to new tasks, reusing knowledge from previous tasks while not forgetting anything about them.  ...  To allow already learned features to adapt to the current task without changing the behavior of these features for previous tasks, we introduce task-specific feature normalization.  ...  Piggyback proposes to use a pretrained network as a backbone and then uses binary masks on the weights to create different sub-networks for each task [9] .  ... 
arXiv:2001.08714v2 fatcat:qosis2hk35bhnnn5ofltqvvyjy

The RGB-D Triathlon: Towards Agile Visual Toolboxes for Robots [article]

Fabio Cermelli, Massimiliano Mancini, Elisa Ricci, Barbara Caputo
2019 arXiv   pre-print
The problem of learning flexible models which can handle multiple tasks in a lightweight manner has recently gained attention in the computer vision community and benchmarks supporting this research have  ...  This is clearly sub-optimal for a robot which is often required to perform simultaneously multiple visual recognition tasks in order to properly act and interact with the environment.  ...  Piggyback (PB) [19] makes use of task-specific binary masks that are added to each convolutional layer of a backbone network.  ... 
arXiv:1904.00912v2 fatcat:xciuehpbm5hbde67fkqbm633ci

Side-Tuning: A Baseline for Network Adaptation via Additive Side Networks [article]

Jeffrey O Zhang, Alexander Sax, Amir Zamir, Leonidas Guibas, Jitendra Malik
2020 arXiv   pre-print
Adaptation can be useful in cases when training data is scarce, when a single learner needs to perform multiple tasks, or when one wishes to encode priors in the network.  ...  When training a neural network for a desired task, one may prefer to adapt a pre-trained network rather than starting from randomly initialized weights.  ...  Acknowledgements: This material is based upon work supported by ONR MURI (N00014-14-1-0671), Vannevar Bush Faculty Fellowship, an Amazon AWS Machine Learning Award, NSF (IIS-1763268), a BDD grant and TRI  ... 
arXiv:1912.13503v4 fatcat:dkbrxaqvffh2vhlatzvvx2ygsu

Neural Network Module Decomposition and Recomposition [article]

Hiroaki Kingetsu, Kenichi Kobayashi, Taiji Suzuki
2021 arXiv   pre-print
We propose a modularization method that decomposes a deep neural network (DNN) into small modules from a functionality perspective and recomposes them into a new model for some other task.  ...  To extract modules, we designed a learning method and a loss function to maximize shared weights among modules.  ...  Piggyback: gence Review, 52(1): 527–561. Adapting a Single Network to Multiple Tasks by Learning Anand, R.; Mehrotra, K.; Mohan, C.  ... 
arXiv:2112.13208v1 fatcat:zcccxh6nmrberacsaj3b5ct4vi

Multi-Task Learning for Dense Prediction Tasks: A Survey [article]

Simon Vandenhende, Stamatios Georgoulis, Wouter Van Gansbeke, Marc Proesmans, Dengxin Dai, Luc Van Gool
2021 IEEE Transactions on Software Engineering   accepted
The typical approach is to learn these tasks in isolation, that is, a separate neural network is trained for each individual task.  ...  Yet, recent multi-task learning (MTL) techniques have shown promising results w.r.t. performance, computations and/or memory footprint, by jointly tackling multiple tasks through a learned shared representation  ...  Piggyback [66] showed how to adapt a single, fixed neural network to a multi-task network by learning binary masks. Huang et al.  ... 
doi:10.1109/tpami.2021.3054719 pmid:33497328 arXiv:2004.13379v3 fatcat:75fbdo4ax5fyvgyi63c7dglkgu

Multi-Task Learning with Deep Neural Networks: A Survey [article]

Michael Crawshaw
2020 arXiv   pre-print
Multi-task learning (MTL) is a subfield of machine learning in which multiple tasks are simultaneously learned by a shared model.  ...  However, the simultaneous learning of multiple tasks presents new design and optimization challenges, and choosing which tasks should be learned jointly is in itself a non-trivial problem.  ...  Piggyback (Mallya et al., 2018 ) is a method for adapting a pre-trained network on a related task by learning to mask out individual weights of the original network.  ... 
arXiv:2009.09796v1 fatcat:d676uupucvgrbgnvsijqcexcqi

Depthwise Convolution Is All You Need for Learning Multiple Visual Domains

Yunhui Guo, Yandong Li, Liqiang Wang, Tajana Rosing
2019 PROCEEDINGS OF THE THIRTIETH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE AND THE TWENTY-EIGHTH INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE  
In this paper, we propose a multi-domain learning architecture based on depthwise separable convolution.  ...  If there exists a universal structure in different visual domains that can be captured via a common parameterization, then we can use a single model for all domains rather than one model per domain.  ...  This work is supported in part by CRISP, one of six centers in JUMP, an SRC program sponsored by DARPA. This work is also supported by NSF CHASE-CI #1730158.  ... 
doi:10.1609/aaai.v33i01.33018368 fatcat:h2qvsjaj4faj5iiddi52utvrie

How fine can fine-tuning be? Learning efficient language models [article]

Evani Radiya-Dixit, Xin Wang
2020 arXiv   pre-print
Given a language model pre-trained on massive unlabeled text corpora, only very light supervised fine-tuning is needed to learn a task: the number of fine-tuning steps is typically five orders of magnitude  ...  As a result, fine-tuning of huge language models can be achieved by simply setting a certain number of entries in certain layers of the pre-trained parameters to zero, saving both task-specific parameter  ...  We also wish to thank Robert S. Schreiber, Urs Köster, Jorge Albericio, Natalia S. Vassilieva, and Marcel Nassar for discussions and feedback on the manuscript.  ... 
arXiv:2004.14129v1 fatcat:qdp5iu4nbraghgpiu5yclbcici
« Previous Showing results 1 — 15 out of 607 results