End-to-End Multilingual Speech Recognition System with Language Supervision Training

Danyang LIU, Ji XU, Pengyuan ZHANG
2020 IEICE transactions on information and systems  
End-to-end (E2E) multilingual automatic speech recognition (ASR) systems aim to recognize multilingual speeches in a unified framework. In the current E2E multilingual ASR framework, the output prediction for a specific language lacks constraints on the output scope of modeling units. In this paper, a language supervision training strategy is proposed with language masks to constrain the neural network output distribution. To simulate the multilingual ASR scenario with unknown language identity
more » ... information, a language identification (LID) classifier is applied to estimate the language masks. On four Babel corpora, the proposed E2E multilingual ASR system achieved an average absolute word error rate (WER) reduction of 2.6% compared with the multilingual baseline system. key words: multilingual speech recognition, language-adaptive training, hybrid attention/CTC
doi:10.1587/transinf.2019edl8214 fatcat:itsc4hdm6rf2tnadzw7vmkb2s4