A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf
.
Adaptive Distillation: Aggregating Knowledge from Multiple Paths for Efficient Distillation
[article]
2021
arXiv
pre-print
Knowledge Distillation is becoming one of the primary trends among neural network compression algorithms to improve the generalization performance of a smaller student model with guidance from a larger teacher model. This momentous rise in applications of knowledge distillation is accompanied by the introduction of numerous algorithms for distilling the knowledge such as soft targets and hint layers. Despite this advancement in different techniques for distilling the knowledge, the aggregation
arXiv:2110.09674v2
fatcat:h7kjo57luvhphpykysp4balpwu