A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Filters
Graph Representation Learning via Multi-task Knowledge Distillation
[article]
2019
arXiv
pre-print
Preserved Fulltext
Specifically, we proposed a multi-task knowledge distillation method. ...
By incorporating network-theory-based graph metrics as auxiliary tasks, we show on both synthetic and real datasets that the proposed multi-task learning method can improve the prediction performance of ...
We then use these graph metrics as auxiliary tasks and distill the knowledge of network theory into the learned graph representations via multi-task learning. ...
arXiv:1911.05700v1
fatcat:afxxmfxj6remjl3ilirfm2gy6u
Better and Faster: Knowledge Transfer from Multiple Self-supervised Learning Tasks via Graph Distillation for Video Classification
[article]
2018
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
In this paper, we propose a graph-based distillation framework to address these problems: (1) We propose logits graph and representation graph to transfer knowledge from multiple self-supervised tasks, ...
where the former distills classifier-level knowledge by solving a multi-distribution joint matching problem, and the latter distills internal feature knowledge from pairwise ensembled representations ...
We first train the teacher models for privileged information learning on auxiliary tasks, then transfer the knowledge from teachers to student via the proposed graph distillation framework. ...
arXiv:1804.10069v1
fatcat:27xmwll4vzec3aatscu6zmwriu
Better and Faster: Knowledge Transfer from Multiple Self-supervised Learning Tasks via Graph Distillation for Video Classification
2018
Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
In this paper, we propose a graph-based distillation framework to address these problems: (1) We propose logits graph and representation graph to transfer knowledge from multiple self-supervised tasks, ...
where the former distills classifier-level knowledge by solving a multi-distribution joint matching problem, and the latter distills internal feature knowledge from pairwise ensembled representations ...
We first train the teacher models for privileged information learning on auxiliary tasks, then transfer the knowledge from teachers to student via the proposed graph distillation framework. ...
doi:10.24963/ijcai.2018/158
dblp:conf/ijcai/ZhangP18a
fatcat:56vujzi3ajhfdhmnxubck7relu
On Representation Knowledge Distillation for Graph Neural Networks
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Knowledge distillation is a promising learning paradigm for boosting the performance and reliability of resource-efficient graph neural networks (GNNs) using more expressive yet cumbersome teacher models ...
Distillation (G-CRD), which uses contrastive learning to align the student node embeddings to those of the teacher in a shared representation space. ...
Graph Representation Learning Graph Neural Networks (GNNs) take as input an unordered set of nodes and the graph connectivity among them, and learn latent node representations for them via iterative feature ...
arXiv:2111.04964v1
fatcat:xowf5oylhnhtpk6lai23cd7aii
Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlooks
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Note that this fulltext copy is not of the "primary" version of this work. The version it corresponds to is:
2020 pre-print arXiv:2004.05937v4 fatcat:v64urq3p5rajdb45cyhxri24qqOther Versions
To achieve faster speeds and to handle the problems caused by the lack of data, knowledge distillation (KD) has been proposed to transfer information learned from one model to another. ...
KD is often characterized by the so-called 'Student-Teacher' (S-T) learning framework and has been broadly applied in model compression and knowledge transfer. ...
[17] considers the instance features and instance relationships as instance graphs, and [213] builds an input graph representation for multi-task knowledge distillation. ...
arXiv:2004.05937v6
fatcat:yqzo7nylzbbn7pfhzpfc2qaxea
Graphonomy: Universal Human Parsing via Graph Transfer Learning
2019
2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
By distilling universal semantic graph representation to each specific task, Graphonomy is able to predict all levels of parsing labels in one system without piling up the complexity. ...
In particular, Graphonomy first learns and propagates compact high-level graph representation among the labels within one dataset via Intra-Graph Reasoning, and then transfers semantic information across ...
Superior to multi-task learning, our Graphonomy is able to distill universal semantic graph representation and enhance individualized representation for each label graph. # Basic network [3] Adjacency ...
doi:10.1109/cvpr.2019.00763
dblp:conf/cvpr/Gong0LS0L19
fatcat:rv5xgqag4jcghmzffi3s67fp2u
Graphonomy: Universal Human Parsing via Graph Transfer Learning
[article]
2019
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
By distilling universal semantic graph representation to each specific task, Graphonomy is able to predict all levels of parsing labels in one system without piling up the complexity. ...
In particular, Graphonomy first learns and propagates compact high-level graph representation among the labels within one dataset via Intra-Graph Reasoning, and then transfers semantic information across ...
Superior to multi-task learning, our Graphonomy is able to distill universal semantic graph representation and enhance individualized representation for each label graph. # Basic network [3] Adjacency ...
arXiv:1904.04536v1
fatcat:di2yce3ytbhadml5lljt7yn66m
Iterative Graph Self-Distillation
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Inspired by the recent success of unsupervised contrastive learning, we aim to learn graph-level representation in an unsupervised manner. ...
Specifically, we propose a novel unsupervised graph learning paradigm called Iterative Graph Self-Distillation (IGSD) which iteratively performs the teacher-student distillation with graph augmentations ...
Table 3 : Effects of projectors on Graph classification accuracies (%).
CONCLUSIONS In this paper, we propose IGSD, a novel graph-level representation learning framework via self-distillation. ...
arXiv:2010.12609v2
fatcat:h5csmfxatbg4jcliukikimsgnm
Knowledge Distillation: A Survey
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Other Versions
As a representative type of model compression and acceleration, knowledge distillation effectively learns a small student model from a large teacher model. ...
This paper provides a comprehensive survey of knowledge distillation from the perspectives of knowledge categories, training schemes, teacher-student architecture, distillation algorithms, performance ...
Multi-head graph-based knowledge distillation was proposed by Lee and Song (2019) . The graph knowledge is the intra-data relations between any two feature maps via multi-head attention network. ...
arXiv:2006.05525v6
fatcat:aedzaeln5zf3jgjsgsn5kvjrri
Joint Multiple Intent Detection and Slot Filling via Self-distillation
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
In this paper, we propose a novel Self-Distillation Joint NLU model (SDJN) for multi-intent NLU. ...
Then, we design an auxiliary loop via self-distillation with three orderly arranged decoders: Initial Slot Decoder, MIL Intent Decoder, and Final Slot Decoder. ...
Second, inspired by self-distillation network [22] , we proposed a self-distillation method for joint NLU modeling by taking advantage of multi-task. ...
arXiv:2108.08042v1
fatcat:6twwj42w2neh3axwxiffu5sh2a
Multi-Scale Contrastive Siamese Networks for Self-Supervised Graph Representation Learning
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Other Versions
to learn node representations by enhancing Siamese self-distillation with multi-scale contrastive learning. ...
Graph representation learning plays a vital role in processing graph-structured data. However, prior arts on graph representation learning heavily rely on labeling information. ...
By leveraging the backbone Siamese GNNs, we design a cross-network contrastiveness to distill the knowledge from historical representations to guide and stabilize the training of online graph encoder. ...
arXiv:2105.05682v2
fatcat:57t7mylspbbs3hb4wf3h5jsgoy
CCGL: Contrastive Cascade Graph Learning
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Other Versions
In this work, we present Contrastive Cascade Graph Learning (CCGL), a novel framework for cascade graph representation learning in a contrastive, self-supervised, and task-agnostic way. ...
Second, it learns a generic model for graph cascade tasks via self-supervised contrastive pre-training using both unlabeled and labeled data. ...
This might be because the model learns the generic knowledge in pre-training and task-specific knowledge in fine-tuning and distillation. ...
arXiv:2107.12576v1
fatcat:kb3si37j65gntar64i63aq7hzi
Knowledge Integration Networks for Action Recognition
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
We explore two pre-trained models as teacher networks to distill the knowledge of human and scene for training the auxiliary tasks of KINet. ...
an Action Knowledge Graph (AKG) for effectively fusing high-level context information. ...
three tasks via knowledge distillation, yet without applying CBI module or Action Knowledge Graph here. ...
arXiv:2002.07471v1
fatcat:mzg6xcv7nbewbb6ungcvinfgwi
Knowledge Integration Networks for Action Recognition
2020
PROCEEDINGS OF THE THIRTIETH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE AND THE TWENTY-EIGHTH INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
We explore two pre-trained models as teacher networks to distill the knowledge of human and scene for training the auxiliary tasks of KINet. ...
an Action Knowledge Graph (AKG) for effectively fusing high-level context information. ...
three tasks via knowledge distillation, yet without applying CBI module or Action Knowledge Graph here. ...
doi:10.1609/aaai.v34i07.6983
fatcat:mb2yttp3xrbppm2ty652ppnbb4
Graphonomy: Universal Image Parsing via Graph Reasoning and Transfer
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
To address these challenges, we propose a graph reasoning and transfer learning framework, named "Graphonomy", which incorporates human knowledge and label taxonomy into the intermediate graph representation ...
The former extracts the semantic graph in each domain to improve the feature representation learning by propagating information with the graph; the latter exploits the dependencies among the graphs from ...
domains via graph transfer learning to achieve multiple levels of human parsing tasks. ...
arXiv:2101.10620v1
fatcat:hnbuqiugsfhvbc7phn5htmsvcy
« Previous
Showing results 1 — 15 out of 6,387 results