The WMT'18 Morpheval test suites for English-Czech, English-German, English-Finnish and Turkish-English

Franck Burlot, Yves Scherrer, Vinit Ravishankar, Ondřej Bojar, Stig-Arne Grönroos, Maarit Koponen, Tommi Nieminen, François Yvon
2018 Proceedings of the Third Conference on Machine Translation: Shared Task Papers  
Progress in the quality of machine translation output calls for new automatic evaluation procedures and metrics. In this paper, we extend the Morpheval protocol introduced by Burlot and Yvon (2017) for the English-to-Czech and English-to-Latvian translation directions to three additional language pairs, and report its use to analyze the results of WMT 2018's participants for these language pairs. Considering additional, typologically varied source and target languages also enables us to draw
more » ... e generalizations regarding this morphology-oriented evaluation procedure.
doi:10.18653/v1/w18-6433 dblp:conf/wmt/BurlotSRBGKNY18 fatcat:5prdnwk4f5bz3l73xcvaghjzbu