Evaluation in the ARPA machine translation program

John S. White, Theresa A. O'Connell
1994 Proceedings of the workshop on Human Language Technology - HLT '94   unpublished
In the second year of evaluations of the ARPA HLT Machine Translation (MT) Initiative, methodologies developed and tested in 1992 were applied to the 1993 MT test runs. The current methodology optimizes the inherently subjective judgments on translation accuracy and quality by channeling the judgments of non-translators into many data points which reflect both the comparison of the performance of the research MT systems with production MT systems and against the performance of novice
more » ... f novice translators. This paper discusses the three evaluation methods used in the 1993 evaluation, the results of the evaluations, and preliminary characterizations of the Winter 1994 evaluation, now underway. The efforts under discussion focus on measuring the.progress of core MT technology and increasing the sensitivity and portability of MT evaluation methodology.
doi:10.3115/1075812.1075840 fatcat:vhjwnsp2mvcs3e54yjmyx2ik6y