This paper addresses the issue of hierarchical relevance determination (HRD), which boils down to determining all degrees of freedom in a supervised mixture distribution automatically. Such relevance determination is useful for wide range of machine learning applications. However, it is difficult to solve the HRD task because its objective function includes L 0 norm such as the number of models in the mixture and the optimal features to use. Our contribution is twofold. As the main

more »

... we formally defined the HRD problem and subsequently proposed a solution strategy, the SICM (sequential in formation criterion minimization) algorithm. The SICM algorithm enables us to continuously minimize an information criterion such as AIC or BIC, both of which includes the number of parameters, and therefore it enables us to determine all the degrees of freedom automatically. As another contribution, we realized a concrete implementation of the ideas, and tested this on actual data. Experiments using a hierarchical model which is constructed using the SICM algorithm have revealed that SICM has capability of constructing interpretable and highly accurate model. 1 There is no representative name for such a task, though it is an existing and well-known problem. Therefore we have given the name HRD in this paper.