Adaptive Risk Minimization: Learning to Adapt to Domain Shift

Marvin Zhang; Henrik Marklund; Nikita Dhawan; Abhishek Gupta; Sergey Levine; Chelsea Finn

Adaptive Risk Minimization: Learning to Adapt to Domain Shift [article]

Marvin Zhang, Henrik Marklund, Nikita Dhawan, Abhishek Gupta, Sergey Levine, Chelsea Finn

<span title="2021-12-01">2021</span> <i > arXiv </i>   <span class="release-stage" >pre-print</span>

A fundamental assumption of most machine learning algorithms is that the training and test data are drawn from the same underlying distribution. However, this assumption is violated in almost all practical applications: machine learning systems are regularly tested under distribution shift, due to changing temporal correlations, atypical end users, or other factors. In this work, we consider the problem setting of domain generalization, where the training data are structured into domains and

more »

... re may be multiple test time shifts, corresponding to new domains or domain distributions. Most prior methods aim to learn a single robust model or invariant feature space that performs well on all domains. In contrast, we aim to learn models that adapt at test time to domain shift using unlabeled test points. Our primary contribution is to introduce the framework of adaptive risk minimization (ARM), in which models are directly optimized for effective adaptation to shift by learning to adapt on the training domains. Compared to prior methods for robustness, invariance, and adaptation, ARM methods provide performance gains of 1-4% test accuracy on a number of image classification problems exhibiting domain shift.

<span class="external-identifiers"> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2007.02931v4">arXiv:2007.02931v4</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/2gjsygbev5cmhdryhrwijvfeju">fatcat:2gjsygbev5cmhdryhrwijvfeju</a> </span>

<span class="ui label basic blue small">Multiple Versions</span>

<a target="_blank" rel="noopener" href="https://web.archive.org/web/20211203025107/https://arxiv.org/pdf/2007.02931v4.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/58/a4/58a4a8e23256e0c9dd5071de0587b84f5f88d8c9.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2007.02931v4" title="arxiv.org access"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> arxiv.org </button> </a>

Adaptive Risk Minimization: Learning to Adapt to Domain Shift [article]

Preserved Fulltext

Other Versions