Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinement

Jason Lee, Elman Mansimov, Kyunghyun Cho
2018 Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing  
We propose a conditional non-autoregressive neural sequence model based on iterative refinement. The proposed model is designed based on the principles of latent variable models and denoising autoencoders, and is generally applicable to any sequence generation task. We extensively evaluate the proposed model on machine translation (En$De and En$Ro) and image caption generation, and observe that it significantly speeds up decoding while maintaining the generation quality comparable to the autoregressive counterpart.
doi:10.18653/v1/d18-1149 dblp:conf/emnlp/LeeMC18 fatcat:abgjrqls65h4zb5hfsyyap3hmy