Disentangled Representations from Non-Disentangled Models [article]

Valentin Khrulkov, Leyla Mirvakhabova, Ivan Oseledets, Artem Babenko
<span title="2021-02-11">2021</span> <i > arXiv </i> &nbsp; <span class="release-stage" >pre-print</span>
Constructing disentangled representations is known to be a difficult task, especially in the unsupervised scenario. The dominating paradigm of unsupervised disentanglement is currently to train a generative model that separates different factors of variation in its latent space. This separation is typically enforced by training with specific regularization terms in the model's objective function. These terms, however, introduce additional hyperparameters responsible for the trade-off between
more &raquo; ... entanglement and generation quality. While tuning these hyperparameters is crucial for proper disentanglement, it is often unclear how to tune them without external supervision. This paper investigates an alternative route to disentangled representations. Namely, we propose to extract such representations from the state-of-the-art generative models trained without disentangling terms in their objectives. This paradigm of post hoc disentanglement employs little or no hyperparameters when learning representations while achieving results on par with existing state-of-the-art, as shown by comparison in terms of established disentanglement metrics, fairness, and the abstract reasoning task. All our code and models are publicly available.
<span class="external-identifiers"> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2102.06204v1">arXiv:2102.06204v1</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/yeg24kjkkzdnhhcxv4igm5mz3u">fatcat:yeg24kjkkzdnhhcxv4igm5mz3u</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20210214042537/https://arxiv.org/pdf/2102.06204v1.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/53/36/5336fe502d440ee3c53c01f1fa46604154eba1a2.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2102.06204v1" title="arxiv.org access"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> arxiv.org </button> </a>