A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Filters
Compressing GANs using Knowledge Distillation
[article]
2019
arXiv
pre-print
Preserved Fulltext
Therefore, we propose a method to compress GANs using knowledge distillation techniques, in which a smaller "student" GAN learns to mimic a larger "teacher" GAN. ...
We show that the distillation methods used on MNIST, CIFAR-10, and Celeb-A datasets can compress teacher GANs at ratios of 1669:1, 58:1, and 87:1, respectively, while retaining the quality of the generated ...
Similarly, we see that the student GANs consistently outper-Compressing GANs using Knowledge Distillation Figure 6 . ...
arXiv:1902.00159v1
fatcat:35ul76wtbjguzmx74vx3rpupc4
PPCD-GAN: Progressive Pruning and Class-Aware Distillation for Large-Scale Conditional GANs Compression
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
We push forward neural network compression research by exploiting a novel challenging task of large-scale conditional generative adversarial networks (GANs) compression. ...
To this end, we propose a gradually shrinking GAN (PPCD-GAN) by introducing progressive pruning residual block (PP-Res) and class-aware distillation. ...
Distill+NAS [6] indicates the model (SA-GAN [2] ) is first trained with knowledge distillation (i.e., learnable remapping [6] ) and then compressed using NAS [17] . ...
arXiv:2203.08456v1
fatcat:oe2dyjbzdjd57jsoersxalloqa
MFAGAN: A Compression Framework for Memory-Efficient On-Device Super-Resolution GAN
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Third, to balance the student discriminator and the compressed generator, we distill both the generator and the discriminator. ...
In this paper, we propose a novel compression framework Multi-scale Feature Aggregation Net based GAN (MFAGAN) for reducing the memory access cost of the generator. ...
Besides the generator, the discriminator stores useful knowledge of a learned GAN-based SR. It is useful to distill the teacher discriminator to stabilize the compressed generator training. ...
arXiv:2107.12679v1
fatcat:66jm5m2q5vaprhirf7weg74u5i
GAN Compression: Efficient Architectures for Interactive Conditional GANs
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
First, to stabilize the GAN training, we transfer knowledge of multiple intermediate representations of the original model to its compressed model, and unify unpaired and paired learning. ...
Directly applying existing CNNs compression methods yields poor performance due to the difficulty of GAN training and the differences in generator architectures. ...
A widely-used method for CNN model compression is knowledge distillation [25, 48, 10, 72, 36, 53, 12] . ...
arXiv:2003.08936v3
fatcat:ng36z3k2hzbfbob5gjrvh62kiq
A survey on GAN acceleration using memory compression techniques
2021
Journal of Engineering and Applied Science (Cairo) (Online)
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Lossy compression techniques are further classified into (a) pruning, (b) knowledge distillation, (c) low-rank factorization, (d) lowering numeric precision, and (e) encoding. ...
Our findings showed the superiority of knowledge distillation over pruning alone and the gaps in the research field that needs to be explored like encoding and different combination of compression techniques ...
Although this work "uses" GAN to perform distillation, it does not consider GAN themselves for compression. ...
doi:10.1186/s44147-021-00045-5
fatcat:hy3oxa4fvzavhophwiekph4rum
GAN Slimming: All-in-One GAN Compression by A Unified Optimization Framework
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
GS seamlessly integrates three mainstream compression techniques: model distillation, channel pruning and quantization, together with the GAN minimax objective, into one unified optimization form, that ...
To this end, we propose the first unified optimization framework combining multiple compression means for GAN compression, dubbed GAN Slimming (GS). ...
Knowledge distillation was first developed in [22] to transfer the knowledge in an ensemble of models to a single model, using a soft target distribution produced by the former models. ...
arXiv:2008.11062v1
fatcat:567bndbj5re2himu2kwwyiz7y4
P-KDGAN: Progressive Knowledge Distillation with GANs for One-class Novelty Detection
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Therefore, Progressive Knowledge Distillation with GANs (PKDGAN) is proposed to learn compact and fast novelty detection networks. ...
The P-KDGAN is a novel attempt to connect two standard GANs by the designed distillation loss for transferring knowledge from the teacher to the student. ...
To compress the model, the progressive knowledge distillation with GANs is proposed, which is a novel exploration that applies the knowledge distillation on two standard GANs. ...
arXiv:2007.06963v2
fatcat:b5xddnfusbat5ci3k2jyuolho4
A Survey on GAN Acceleration Using Memory Compression Technique
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Because data transfer is the main source of energy usage, memory compression leads to the most savings. Thus, in this paper, we survey memory compression techniques for CNN-Based GANs. ...
Hence, accelerating GANs is pivotal. Accelerating GANs can be classified into three main tracks: (1) Memory compression, (2) Computation optimization, and (3) Data-flow optimization. ...
Although this work "uses" GAN to perform distillation, it does not consider GAN themselves for compression. ...
arXiv:2108.06626v1
fatcat:b4imro6ap5fkvoceewn3qbazgy
Online Multi-Granularity Distillation for GAN Compression
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Other Versions
We offer the first attempt to popularize single-stage online distillation for GAN-oriented compression, where the progressively promoted teacher generator helps to refine the discriminator-free based student ...
Although recent efforts on compressing GANs have acquired remarkable results, they still exist potential model redundancies and can be further compressed. ...
Knowledge Distillation Knowledge Distillation (KD) [19] is a fundamental compression technique, where a smaller student model is optimized under the effective information transfer and supervision of ...
arXiv:2108.06908v2
fatcat:56ctz3tbofatjaqafmvnxx6lnq
Closing the Generalization Gap of Adaptive Gradient Methods in Training Deep Neural Networks
2020
Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
To compress the model, the progressive knowledge distillation with GANs is proposed, which is a novel exploration that applies the knowledge distillation on two standard GANs. ...
Moreover, our proposed method can be used to compress other GANs-based applications, such as image generation. ...
doi:10.24963/ijcai.2020/448
dblp:conf/ijcai/ZhangCS20
fatcat:lo7lunpacnbppirpuu3l2zp3ha
Distilling Portable Generative Adversarial Networks for Image Translation
2020
PROCEEDINGS OF THE THIRTIETH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE AND THE TWENTY-EIGHTH INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Inspired by knowledge distillation, a student generator of fewer parameters is trained by inheriting the low-level and high-level information from the original heavy teacher generator. ...
Despite Generative Adversarial Networks (GANs) have been widely used in various image-to-image translation tasks, they can be hardly applied on mobile devices due to their heavy computation and storage ...
Moreover, they do not distill knowledge to the discriminator, which takes an important part in GANs' training. ...
doi:10.1609/aaai.v34i04.5765
fatcat:v3stk3d36jdxpn4xcu4ly2y65u
Distilling portable Generative Adversarial Networks for Image Translation
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Inspired by knowledge distillation, a student generator of fewer parameters is trained by inheriting the low-level and high-level information from the original heavy teacher generator. ...
Despite Generative Adversarial Networks (GANs) have been widely used in various image-to-image translation tasks, they can be hardly applied on mobile devices due to their heavy computation and storage ...
Moreover, they do not distill knowledge to the discriminator, which takes an important part in GANs' training. ...
arXiv:2003.03519v1
fatcat:zch5iumcvbbofo6eneoiwv75hm
Semantic Relation Preserving Knowledge Distillation for Image-to-Image Translation
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Other Versions
In this work, we propose a novel method to address this problem by applying knowledge distillation together with distillation of a semantic relation preserving matrix. ...
In contrast to existing compression methods designed for classification tasks, our proposed method adapts well to the image-to-image translation task on GANs. ...
Conclusions We approach model compression of GANs via a novel proposed method extended on traditional knowledge distillation. ...
arXiv:2104.15082v2
fatcat:dbcz2zs3xbcjthh63kztcpuexm
Region-aware Knowledge Distillation for Efficient Image-to-Image Translation
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
In this paper, we propose Region-aware Knowledge Distillation ReKo to compress image-to-image translation models. ...
To address this issue, knowledge distillation is proposed to transfer the knowledge from a cumbersome teacher model to an efficient student model. ...
GAN Knowledge Distillation In the last several years, there has been some research proposed to apply knowledge distillation to the compression of GANs. 3 Methodology
Formulation Patch-wise Contrastive ...
arXiv:2205.12451v1
fatcat:gzifgo4zobfovf4lqtu4i7m5ue
Teachers Do More Than Teach: Compressing Image-to-Image Models
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Finally, we propose to distill knowledge through maximizing feature similarity between teacher and student via an index named Global Kernel Alignment (GKA). ...
Recent efforts on compression GANs show noticeable progress in obtaining smaller generators by sacrificing image quality or involving a time-consuming searching process. ...
Most GAN compression methods [1, 9, 20] use response-based distillation, enforcing the synthesized images from the teacher and student networks to be the same. Li et al. ...
arXiv:2103.03467v2
fatcat:d3rjuwhsdbbsvfna53i3vikmbi
« Previous
Showing results 1 — 15 out of 1,972 results