Filters








14,456 Hits in 4.6 sec

On Using Very Large Target Vocabulary for Neural Machine Translation [article]

Sébastien Jean, Kyunghyun Cho, Roland Memisevic, Yoshua Bengio
2015 arXiv   pre-print
In this paper, we propose a method that allows us to use a very large target vocabulary without increasing training complexity, based on importance sampling.  ...  Furthermore, when we use the ensemble of a few models with very large target vocabularies, we achieve the state-of-the-art translation performance (measured by BLEU) on the English->German translation  ...  We acknowledge the support of the following agencies for research funding and computing support: NSERC, Calcul Québec, Compute Canada, the Canada Research Chairs and CIFAR.  ... 
arXiv:1412.2007v2 fatcat:5t2yyc6glrgmrce4a4zo3fedfi

On Using Very Large Target Vocabulary for Neural Machine Translation

Sébastien Jean, Kyunghyun Cho, Roland Memisevic, Yoshua Bengio
2015 Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)  
In this paper, we propose a method based on importance sampling that allows us to use a very large target vocabulary without increasing training complexity.  ...  Furthermore, when we use an ensemble of a few models with very large target vocabularies, we achieve performance comparable to the state of the art (measured by BLEU) on both the English→German and English  ...  We acknowledge the support of the following agencies for research funding and computing support: NSERC, Calcul Québec, Compute Canada, the Canada Research Chairs, CIFAR and Samsung.  ... 
doi:10.3115/v1/p15-1001 dblp:conf/acl/JeanCMB15 fatcat:7a4ljk2lxvfcbkhn4fm7suxrqq

Vocabulary Manipulation for Neural Machine Translation

Haitao Mi, Zhiguo Wang, Abe Ittycheriah
2016 Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)  
In order to capture rich language phenomena, neural machine translation models have to use a large vocabulary size, which requires high computing time and large memory usage.  ...  Our method simply takes into account the translation options of each word or phrase in the source sentence, and picks a very small target vocabulary for each sentence based on a wordto-word translation  ...  Acknowledgment We thank the anonymous reviewers for their comments.  ... 
doi:10.18653/v1/p16-2021 dblp:conf/acl/MiWI16 fatcat:vwxwlv2imvbgznqrai33idvtgy

Vocabulary Manipulation for Neural Machine Translation [article]

Haitao Mi and Zhiguo Wang and Abe Ittycheriah
2016 arXiv   pre-print
In order to capture rich language phenomena, neural machine translation models have to use a large vocabulary size, which requires high computing time and large memory usage.  ...  Our method simply takes into account the translation options of each word or phrase in the source sentence, and picks a very small target vocabulary for each sentence based on a word-to-word translation  ...  Acknowledgment We thank the anonymous reviewers for their comments.  ... 
arXiv:1605.03209v1 fatcat:hzihry5zpzegnbllwobnt4mihi

Significant Enhancements in Machine Translation by Various Deep Learning Approaches

Alpana Upadhyay
2017 American Journal of Computer Science and Information Technology  
It describes and includes all the topics like integrating deep learning in statistical machine translation, developing end-to-end neural machine translation systems, introducing deep learning in machine  ...  Deep learning was first acquainting with Machine Translation in the standard statistical systems. This paper addresses the progress of introduction of deep learning in machine translation.  ...  Dealing with the large softmax normalization on the output that is reliant on the size of target vocabulary, is the main problem with neural machine translation.  ... 
doi:10.21767/2349-3917.100008 fatcat:sc4mccb7ofdhjkhcka5c4jraza

Translation Mechanism of Neural Machine Algorithm for Online English Resources

Yanping Ye, Wei Wang
2021 Complexity  
vocabulary based on the vocabulary alignment information to guide the neural machine translation decoder to more accurately estimate its vocabulary in the target language.  ...  This paper proposes a framework that integrates vocabulary alignment structure for neural machine translation at the vocabulary level.  ...  vocabulary recommendation module is responsible for perceiving and using the attention information of neural machine translation and neural machine translation target language to generate historical information  ... 
doi:10.1155/2021/5564705 fatcat:t2kizmktwbfhjogelf7hqoafuy

Vocabulary Selection Strategies for Neural Machine Translation [article]

Gurvan L'Hostis, David Grangier, Michael Auli
2016 arXiv   pre-print
This brings the time to decode with a state of the art neural translation system to just over 140 msec per sentence on a single CPU core for English-German.  ...  Recent work on improving the efficiency of neural translation models adopted a similar strategy by restricting the output vocabulary to a subset of likely candidates given the source.  ...  Estimating this conditional distribution is linear in the size of the target vocabulary which can be very large for many language pairs (Grave et al., 2016) .  ... 
arXiv:1610.00072v1 fatcat:53jgt76l45f6dhw3sctrmtupbm

Montreal Neural Machine Translation Systems for WMT'15

Sébastien Jean, Orhan Firat, Kyunghyun Cho, Roland Memisevic, Yoshua Bengio
2015 Proceedings of the Tenth Workshop on Statistical Machine Translation  
Neural machine translation (NMT) systems have recently achieved results comparable to the state of the art on a few translation tasks, including English→French and English→German.  ...  We also leverage some of the recent developments in NMT, including the use of large vocabularies, unknown word replacement and, to a limited degree, the inclusion of monolingual language models.  ...  We acknowledge the support of the following agencies for research funding and computing support: NSERC, Calcul Québec, Compute Canada, the Canada Research Chairs, CIFAR, Samsung and TUBITAK.  ... 
doi:10.18653/v1/w15-3014 dblp:conf/wmt/JeanFCMB15 fatcat:4mkwi2d5czg2teiolyzawm5amu

Addressing the Rare Word Problem in Neural Machine Translation

Thang Luong, Ilya Sutskever, Quoc Le, Oriol Vinyals, Wojciech Zaremba
2015 Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)  
Neural Machine Translation (NMT) is a new approach to machine translation that has shown promising results that are comparable to traditional approaches.  ...  We train an NMT system on data that is augmented by the output of a word alignment algorithm, allowing the NMT system to emit, for each OOV word in the target sentence, the position of its corresponding  ...  The first author especially thanks Chris Manning and the Stanford NLP group for helpful comments on the early drafts of the paper.  ... 
doi:10.3115/v1/p15-1002 dblp:conf/acl/LuongSLVZ15 fatcat:uqwi47iddvdpxgssinmhwa5mny

Beyond Word-based Language Model in Statistical Machine Translation [article]

Jiajun Zhang, Shujie Liu, Mu Li, Ming Zhou, Chengqing Zong
2015 arXiv   pre-print
Language model is one of the most important modules in statistical machine translation and currently the word-based language model dominants this community.  ...  However, many translation models (e.g. phrase-based models) generate the target language sentences by rendering and compositing the phrases rather than the words.  ...  For the first four lines, we just use one language model in the translation system.  ... 
arXiv:1502.01446v1 fatcat:ejtqo2fezjdehapy4ikqpuch3e

Neural vs. Phrase-Based Machine Translation in a Multi-Domain Scenario

M. Amin Farajian, Marco Turchi, Matteo Negri, Nicola Bertoldi, Marcello Federico
2017 Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers  
State-of-the-art neural machine translation (NMT) systems are generally trained on specific domains by carefully selecting the training sets and applying proper domain adaptation techniques.  ...  We compare the performance of a generic NMT system and phrase-based statistical machine translation (PBMT) system by training them on a generic parallel corpus composed of data from different domains.  ...  However, all their experiments are performed on one single domain for which there exists a very large training corpus.  ... 
doi:10.18653/v1/e17-2045 dblp:conf/eacl/NegriTFBF17 fatcat:eofrfbqjf5c3jmz6nzij7nb7by

Open Vocabulary Learning for Neural Chinese Pinyin IME [article]

Zhuosheng Zhang and Yafang Huang and Hai Zhao
2019 arXiv   pre-print
To alleviate such inconveniences, we propose a neural P2C conversion model augmented by an online updated vocabulary with a sampling mechanism to support open vocabulary learning during IME working.  ...  Our experiments show that the proposed method outperforms commercial IMEs and state-of-the-art traditional models on standard corpus and true inputting history dataset in terms of multiple metrics and  ...  Estimating this conditional distribution is linear in the size of the target vocabulary which can be very large for many language pairs.  ... 
arXiv:1811.04352v4 fatcat:cyc3sx6bdbejlerxylfemb3j5e

Addressing the Rare Word Problem in Neural Machine Translation [article]

Minh-Thang Luong, Ilya Sutskever, Quoc V. Le, Oriol Vinyals, Wojciech Zaremba
2015 arXiv   pre-print
Neural Machine Translation (NMT) is a new approach to machine translation that has shown promising results that are comparable to traditional approaches.  ...  We train an NMT system on data that is augmented by the output of a word alignment algorithm, allowing the NMT system to emit, for each OOV word in the target sentence, the position of its corresponding  ...  Our system outperforms the current best end-to-end neural machine translation system by the large margin of 2.7 BLEU points.  ... 
arXiv:1410.8206v4 fatcat:pw3chx5vufh35npqf52uxjbxse

Speeding Up Neural Machine Translation Decoding by Shrinking Run-time Vocabulary

Xing Shi, Kevin Knight
2017 Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)  
We speed up Neural Machine Translation (NMT) decoding by shrinking run-time target vocabulary. We experiment with two shrinking approaches: Locality Sensitive Hashing (LSH) and word alignments.  ...  We also report a negative result for LSH on GPUs, due to relatively large overhead, though it was successful on CPUs.  ...  For each word f in the source vocabulary of the neural machine translation model, store the top M target words according to P(e|f ) 4.  ... 
doi:10.18653/v1/p17-2091 dblp:conf/acl/ShiK17 fatcat:dvp2majbynfrveuzw44wgvl7ky

Development of a Recurrent Neural Network Model for English to Yorùbá Machine Translation

Adebimpe Esan, John Oladosu, Christopher Oyeleye, Ibrahim Adeyanju, Olatayo Olaniyan, Nnamdi Okomba, Bolaji Omodunbi, Opeyemi Adanigbo
2020 International Journal of Advanced Computer Science and Applications  
This research developed a recurrent neural network model for English to Yoruba machine translation. Parallel corpus was obtained from the English and Yoruba bible corpus.  ...  The developed model was tested and evaluated using both manual and automatic evaluation techniques. Results from manual evaluation by ten human evaluators show that the system is adequate and fluent.  ...  Decoding was efficiently done using a very large target vocabulary by selecting a small portion of the target vocabulary.  ... 
doi:10.14569/ijacsa.2020.0110574 fatcat:p77k2ddg7fbozi2w3kcnpr2koy
« Previous Showing results 1 — 15 out of 14,456 results