Reliable and Scalable Variational Inference for the Hierarchical Dirichlet Process

Michael C. Hughes, Dae Il Kim, Erik B. Sudderth
2015 International Conference on Artificial Intelligence and Statistics  
We introduce a new variational inference objective for hierarchical Dirichlet process admixture models. Our approach provides novel and scalable algorithms for learning nonparametric topic models of text documents and Gaussian admixture models of image patches. Improving on the point estimates of topic probabilities used in previous work, we define full variational posteriors for all latent variables and optimize parameters via a novel surrogate likelihood bound. We show that this approach has
more » ... rucial advantages for data-driven learning of the number of topics. Via merge and delete moves that remove redundant or irrelevant topics, we learn compact and interpretable models with less computation. Scaling to millions of documents is possible using stochastic or memoized variational updates.
dblp:conf/aistats/HughesKS15 fatcat:ab3kwtvtg5agjoczlzht7o6nfa