Filters








8 Hits in 1.0 sec

SimulEval: An Evaluation Toolkit for Simultaneous Translation [article]

Xutai Ma, Mohammad Javad Dousti, Changhan Wang, Jiatao Gu, Juan Pino
2020 arXiv   pre-print
Therefore, we present SimulEval, an easy-to-use and general evaluation toolkit for both simultaneous text and speech translation.  ...  SimulEval has already been extensively used for the IWSLT 2020 shared task on simultaneous speech translation. Code will be released upon publication.  ...  Conclusion In this paper, we introduced SIMULEVAL, a general and easy-to-use evaluation toolkit for simultaneous speech and text translation.  ... 
arXiv:2007.16193v1 fatcat:j7zkmzmqxvei5gx5ect3f6pucm

SIMULEVAL: An Evaluation Toolkit for Simultaneous Translation

Xutai Ma, Mohammad Javad Dousti, Changhan Wang, Jiatao Gu, Juan Pino
2020 Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations   unpublished
Therefore, we present SIMULEVAL, an easy-to-use and general evaluation toolkit for both simultaneous text and speech translation.  ...  SIMULEVAL has already been extensively used for the IWSLT 2020 shared task on simultaneous speech translation. Code will be released upon publication. 1  ...  Conclusion In this paper, we introduced SIMULEVAL, a general and easy-to-use evaluation toolkit for simultaneous speech and text translation.  ... 
doi:10.18653/v1/2020.emnlp-demos.19 fatcat:l2r6qwyrs5gc5hdun2bctfkzb4

Streaming Simultaneous Speech Translation with Augmented Memory Transformer [article]

Xutai Ma, Yongqiang Wang, Mohammad Javad Dousti, Philipp Koehn, Juan Pino
2020 arXiv   pre-print
We conduct an empirical evaluation of the proposed model on segment, context and memory sizes and we compare our approach to a transformer with a unidirectional mask.  ...  Nevertheless, most of the previous work on simultaneous speech translation, the task of generating translations from partial audio input, ignores the time spent in generating the translation when analyzing  ...  Translation quality is evaluated with case-sensitive detokenized BLEU with SACREBLEU 3 . The latency is evaluated by Average Lagging [3, 7] , with the SimulEval toolkit 4 .  ... 
arXiv:2011.00033v1 fatcat:i5jzmqvhmnhpthl62u4tok4c4m

UniST: Unified End-to-end Model for Streaming and Non-streaming Speech Translation [article]

Qianqian Dong, Yaoming Zhu, Mingxuan Wang, Lei Li
2021 arXiv   pre-print
We will make our codes and evaluation tools publicly available.  ...  This paper presents a unified end-to-end frame-work for both streaming and non-streamingspeech translation.  ...  The latency metrics for streaming UniST is evaluated by AL, DAL and AP based on the SimulEval toolkit 5 .  ... 
arXiv:2109.07368v2 fatcat:bnb2mawdhzgwhdaz4h2l74r774

fairseq S2T: Fast Speech-to-Text Modeling with fairseq [article]

Changhan Wang, Yun Tang, Xutai Ma, Anne Wu, Dmytro Okhonko, Juan Pino
2020 arXiv   pre-print
We introduce fairseq S2T, a fairseq extension for speech-to-text (S2T) modeling tasks such as end-to-end speech recognition and speech-to-text translation.  ...  Fairseq's machine translation models and language models can be seamlessly integrated into S2T workflows for multi-task learning or transfer learning.  ...  Tied multitask learning for neural speech translation.  ... 
arXiv:2010.05171v1 fatcat:tcdojkewtjfyhghjsbogbpgopq

An Empirical Study of End-to-end Simultaneous Speech Translation Decoding Strategies [article]

Ha Nguyen, Yannick Estève, Laurent Besacier
2021 arXiv   pre-print
This paper proposes a decoding strategy for end-to-end simultaneous speech translation.  ...  Our best decoding settings achieve comparable results with a strong cascade model evaluated on the simultaneous translation track of IWSLT 2020 shared task.  ...  Evaluation of simultaneous speech translation In order to measure the latency, we use an adaptive version [17] of Average Lagging (AL) introduced by [3] .  ... 
arXiv:2103.03233v1 fatcat:47j5s2qqqngtlpuqlj2qbjjtzq

The Volctrans Neural Speech Translation System for IWSLT 2021 [article]

Chengqi Zhao and Zhicheng Liu and Jian Tong and Tao Wang and Mingxuan Wang and Rong Ye and Qianqian Dong and Jun Cao and Lei Li
2021 arXiv   pre-print
For text-to-text simultaneous translation, we explore the best practice to optimize the wait-k model.  ...  For text-to-text simultaneous translation, we explore the best practice to optimize the wait-k model.  ...  When evaluating the simultaneous translation, we use the official SimulEval toolkit and report case-sensitive detokenized BLEU (Post, 2018) and Average Lagging (Ma et al., 2019) on MuST-C tst-COMMON  ... 
arXiv:2105.07319v2 fatcat:nu44s3x37fbhdl6dulqa5ntmma

SLTEV: Comprehensive Evaluation of Spoken Language Translation

Ebrahim Ansari, Ondřej Bojar, Barry Haddow, Mohammad Mahmoudi
2021 Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations   unpublished
Spoken Language Translation (SLT), especially when simultaneous, needs to consider additional criteria and does not have a standard evaluation procedure and a widely used toolkit.  ...  To fill the gap, we introduce SLTEV, an open-source tool for assessing SLT in a comprehensive way.  ...  The closest to our work is SIMULEVAL , a client-server toolkit measuring the latency of SLT including any network effects between the evaluated system (client) and the mock user (SIMULEVAL server).  ... 
doi:10.18653/v1/2021.eacl-demos.9 fatcat:odtmtgtsgjbk3bxaqkuyogkldi