Automatic model complexity control using marginalized discriminative growth functions

X. Liu, M.J.F. Gales
2003 IEEE Workshop on Automatic Speech Recognition and Understanding (IEEE Cat. No.03EX721)  
Selecting the model structure with the "appropriate" complexity, is a standard problem for training large vocabulary continuous speech recognition (LVCSR) systems. State-of-the-art LVCSR systems are highly complex. A wide variety of techniques may be used which alter the system complexity and word error rate (WER). Explicitly evaluating systems for all possible configurations is infeasible, hence an automatic model complexity control criterion is highly desirable. Most existing complexity
more » ... l schemes can be classified into two types, Bayesian learning techniques and information theory approaches. An implicit assumption is made in both, that increasing the likelihood on held-out data decreases the WER. However this correlation has been found quite weak for current speech recognition systems. This paper presents a novel discriminative complexity control technique, the marginalization of a discriminative growth function. This is a closer approximation to the true WER than standard approaches. Experimental results on a standard LVCSR Switchboard task, showed that marginalized discriminative growth functions outperforms manually tuned systems and conventional complexity control techniques, such as BIC, in terms of WER.
doi:10.1109/asru.2003.1318400 fatcat:fzs5zwbua5ek3m44ioc2jhugra