Multi-Task Learning in Deep Neural Networks for Mandarin-English Code-Mixing Speech Recognition

Mengzhe CHEN, Jielin PAN, Qingwei ZHAO, Yonghong YAN
2016 IEICE transactions on information and systems  
Multi-task learning in deep neural networks has been proven to be effective for acoustic modeling in speech recognition. In the paper, this technique is applied to Mandarin-English code-mixing recognition. For the primary task of the senone classification, three schemes of the auxiliary tasks are proposed to introduce the language information to networks and improve the prediction of language switching. On the realworld Mandarin-English test corpus in mobile voice search, the proposed schemes
more » ... hanced the recognition on both languages and reduced the relative overall error rates by 3.5%, 3.8% and 5.8% respectively. key words: multi-task learning, deep neural network, Mandarin-English code mixing, speech recognition
doi:10.1587/transinf.2016sll0004 fatcat:rsoewvqt3jhmbfckaclfsks4pa