Most languages have no established writing system and minimal written records. However, textual data is essential for natural language processing, and particularly important for training language models that would facilitate speech recognition. However, bilingual lexicons are often available, since creating lexicons is a fundamental task of documentary linguistics. We investigate the use of such lexicons to improve language models, when textual training data is limited to as few as a thousand

more »

... ntences. The method involves learning cross-lingual word embeddings as a preliminary step in training monolingual language models. Results across a number of languages show that language models are improved by this pre-training. Application to Yongning Na, a threatened language, highlights challenges in deploying the approach in real low-resource environments.