Internet Archive Scholar

Online Learning over Time in Adaptive Neural Machine Translation

Thierry Etchegoyhen, Vicomtech Foundation, Basque Research and Technology Alliance (BRTA), Spain, David Ponce, Harritxu Gete Ugarte, Victor Ruiz Gómez, Vicomtech Foundation, Basque Research and Technology Alliance (BRTA), Spain, Vicomtech Foundation, Basque Research and Technology Alliance (BRTA), Spain, Vicomtech Foundation, Basque Research and Technology Alliance (BRTA), Spain
2021 Proceedings of the Conference Recent Advances in Natural Language Processing - Deep Learning for Natural Language Processing Methods and Applications   unpublished

Preserved Fulltext

fulltext thumbnail
Web Archive Capture PDF (624.5 kB)
https://web.archive.org/web/20211113034405/https://acl-bg.org/proceedings/2021/RANLP%202021/pdf/2021.ranlp-1.47.pdf

A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL. The file type is application/pdf.

Adaptive Machine Translation purports to dynamically include user feedback to improve translation quality. In a post-editing scenario, user corrections of machine translation output are thus continuously incorporated into translation models, reducing or eliminating repetitive error editing and increasing the usefulness of automated translation. In neural machine translation, this goal may be achieved via online learning approaches, where network parameters are updated based on each new sample.
more » ... his type of adaptation typically requires higher learning rates, which can affect the quality of the models over time. Alternatively, less aggressive online learning setups may preserve model stability, at the cost of reduced adaptation to user-generated corrections. In this work, we evaluate different online learning configurations over time, measuring their impact on user-generated samples, as well as separate in-domain and out-of-domain datasets. Results in two different domains indicate that mixed approaches combining online learning with periodic batch fine-tuning might be needed to balance the benefits of online learning with model stability.
doi:10.26615/978-954-452-072-4_047 fatcat:nhgoma5qzbf7zovcq3bwnkeaji
Citation

Thierry Etchegoyhen, Vicomtech Foundation, Basque Research and Technology Alliance (BRTA), Spain, David Ponce, Harritxu Gete Ugarte, Victor Ruiz Gómez, Vicomtech Foundation, Basque Research and Technology Alliance (BRTA), Spain, Vicomtech Foundation, Basque Research and Technology Alliance (BRTA), Spain, Vicomtech Foundation, Basque Research and Technology Alliance (BRTA), Spain. "Online Learning over Time in Adaptive Neural Machine Translation." Proceedings of the Conference Recent Advances in Natural Language Processing - Deep Learning for Natural Language Processing Methods and Applications (2021)