Learning a meta-level prior for feature relevance from multiple related tasks

Su-In Lee, Vassil Chatalbashev, David Vickrey, Daphne Koller
2007 Proceedings of the 24th international conference on Machine learning - ICML '07  
In many prediction tasks, selecting relevant features is essential for achieving good generalization performance. Most feature selection algorithms consider all features to be a priori equally likely to be relevant. In this paper, we use transfer learning -learning on an ensemble of related tasks -to construct an informative prior on feature relevance. We assume that features themselves have meta-features that are predictive of their relevance to the prediction task, and model their relevance
more » ... a function of the meta-features using hyperparameters (called meta-priors). We present a convex optimization algorithm for simultaneously learning the meta-priors and feature weights from an ensemble of related prediction tasks that share a similar relevance structure. Our approach transfers the meta-priors among different tasks, allowing it to deal with settings where tasks have non-overlapping features or where feature relevance varies over the tasks. We show that transfer learning of feature relevance improves performance on two real data sets which illustrate such settings: (1) predicting ratings in a collaborative filtering task, and (2) distinguishing arguments of a verb in a sentence.
doi:10.1145/1273496.1273558 dblp:conf/icml/LeeCVK07 fatcat:peh5k4pmdrhsrdmqd7s54cfgdu