Matrix updates for perceptron training of continuous density hidden Markov models

Chih-Chieh Cheng, Fei Sha, Lawrence K. Saul
2009 Proceedings of the 26th Annual International Conference on Machine Learning - ICML '09  
In this paper, we investigate a simple, mistakedriven learning algorithm for discriminative training of continuous density hidden Markov models (CD-HMMs). Most CD-HMMs for automatic speech recognition use multivariate Gaussian emission densities (or mixtures thereof) parameterized in terms of their means and covariance matrices. For discriminative training of CD-HMMs, we reparameterize these Gaussian distributions in terms of positive semidefinite matrices that jointly encode their mean and
more » ... riance statistics. We show how to explore the resulting parameter space in CD-HMMs with perceptron-style updates that minimize the distance between Viterbi decodings and target transcriptions. We experiment with several forms of updates, systematically comparing the effects of different matrix factorizations, initializations, and averaging schemes on phone accuracies and convergence rates. We present experimental results for context-independent CD-HMMs trained in this way on the TIMIT speech corpus. Our results show that certain types of perceptron training yield consistently significant and rapid reductions in phone error rates.
doi:10.1145/1553374.1553394 dblp:conf/icml/ChengSS09 fatcat:fy66c5mkcngw3lnfbwxmd4hkhq