Adaptive and Dynamic Knowledge Transfer in Multi-task Learning with Attention Networks [chapter]

Tao Ma, Ying Tan
2020 Communications in Computer and Information Science  
Multi-task learning has shown promising results in many applications of machine learning: given several related tasks, it aims to generalize better on the original tasks, by leveraging the knowledge among tasks. The knowledge transfer mainly depends on task relationships. Most of existing multi-task learning methods guide learning processes based on predefined task relationships. However, the associated relationships have not been fully exploited in these methods. Replacing predefined task
more » ... ionships with the adaptively learned ones may lead to superior performance as it can avoid the misguiding of improper predefinition. Therefore, in this paper, we propose Task Relation Attention Networks to adaptively model the task relationships and dynamically control the positive and negative knowledge transfer for different samples in multi-task learning. To evaluate the effectiveness of the proposed method, experiments on various datasets are conducted. The experimental results demonstrate that the proposed method outperforms both classical and state-of-the-art multi-task learning baselines.
doi:10.1007/978-981-15-7205-0_1 fatcat:n33tj4hepnb4vlwpcjhv7fkzuy