Fast and Extensible Phrase Scoring for Statistical Machine Translation

Christian Hardmeier
2010 Prague Bulletin of Mathematical Linguistics  
Existing tools for generating phrase tables for phrase-based Statistical Machine Translation (SMT) are generally optimised towards low memory use to allow processing of large corpora with limited memory. Whilst being a reasonable design choice, this approach does not make optimal use of resources when the sufficient memory is available. We present memscore, a new open-source tool to score phrases in memory. Besides acting as a faster drop-in replacement for existing software, it implements a
more » ... ber of standard smoothing techniques and provides a platform for easy experimentation with new scoring methods.
doi:10.2478/v10108-010-0007-5 fatcat:szkiijxwnvaurbesqe44u7z3ly