Improved word alignment with statistics and linguistic heuristics

Ulf Hermjakob
2009 Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing Volume 1 - EMNLP '09   unpublished
We present a method to align words in a bitext that combines elements of a traditional statistical approach with linguistic knowledge. We demonstrate this approach for Arabic-English, using an alignment lexicon produced by a statistical word aligner, as well as linguistic resources ranging from an English parser to heuristic alignment rules for function words. These linguistic heuristics have been generalized from a development corpus of 100 parallel sentences. Our aligner, UALIGN, outperforms
more » ... oth the commonly used GIZA++ aligner and the state-of-theart LEAF aligner on F-measure and produces superior scores in end-to-end statistical machine translation, +1.3 BLEU points over GIZA++, and +0.7 over LEAF.
doi:10.3115/1699510.1699540 fatcat:ngocj46bdzbhzdhf2gb6jhmgbm