Filters








127,895 Hits in 4.5 sec

Discrete Variational Attention Models for Language Generation [article]

Xianghong Fang and Haoli Bai and Zenglin Xu and Michael Lyu and Irwin King
2021 arXiv   pre-print
To tackle these issues, in this paper we propose the discrete variational attention model with categorical distribution over the attention mechanism owing to the discrete nature in languages.  ...  Variational autoencoders have been widely applied for natural language generation, however, there are two long-standing problems: information under-representation and posterior collapse.  ...  Conclusion In this paper, we propose discrete variational attention model, a new algorithm for natural language generation.  ... 
arXiv:2004.09764v4 fatcat:4ipk5qpxynhzxer4nbatkfhhr4

Discrete Auto-regressive Variational Attention Models for Text Modeling [article]

Xianghong Fang and Haoli Bai and Jian Li and Zenglin Xu and Michael Lyu and Irwin King
2021 arXiv   pre-print
We further design discrete latent space for the variational attention and mathematically show that our model is free from posterior collapse.  ...  In this paper, we propose Discrete Auto-regressive Variational Attention Model (DAVAM) to address the challenges.  ...  CONCLUSION In this paper, we propose the discrete auto-regressive variational attention model, a new deep generative model for text modeling.  ... 
arXiv:2106.08571v1 fatcat:hindfinnnbho3ixpoaiz4aqfi4

Variational Attention Using Articulatory Priors for Generating Code Mixed Speech Using Monolingual Corpora

SaiKrishna Rallabandi, Alan W. Black
2019 Interspeech 2019  
Subjective evaluation shows that our systems are capable of generating high quality synthesis in code mixed scenarios.  ...  We subject the prior distribution for such latent variables to match articulatory constraints.  ...  Acknowledgements We thank AFRL for funding this research, student volunteers for taking part in the listening evaluations and reviewers for such valuable feedback.  ... 
doi:10.21437/interspeech.2019-1103 dblp:conf/interspeech/RallabandiB19 fatcat:td4tbtiy6fab3h62lzilzav6be

Morphological Inflection Generation with Multi-space Variational Encoder-Decoders

Chunting Zhou, Graham Neubig
2017 Proceedings of the CoNLL SIGMORPHON 2017 Shared Task: Universal Morphological Reinflection  
The system is based on the multi-space variational encoder-decoder (MSVED) method of Zhou and Neubig (2017), which employs both continuous and discrete latent variables for the variational encoder-decoder  ...  We discuss some language-specific errors and present result analysis.  ...  We thank Matthew Honnibal for pointing out that the data distribution of Wikipedia corpus might be biased.  ... 
doi:10.18653/v1/k17-2005 dblp:conf/conll/ZhouN17 fatcat:4qdlr6iearbnpenlcedqhuojj4

Multi-space Variational Encoder-Decoders for Semi-supervised Labeled Sequence Transduction [article]

Chunting Zhou, Graham Neubig
2017 arXiv   pre-print
The generative model can use neural networks to handle both discrete and continuous latent variables to exploit various features of data.  ...  On the SIGMORPHON morphological inflection benchmark, our model outperforms single-model state-of-art results by a large margin for the majority of languages.  ...  Acknowledgments The authors thank Jiatao Gu, Xuezhe Ma, Zihang Dai and Pengcheng Yin for their helpful discussions. This work has been supported in part by an Amazon Academic Research Award.  ... 
arXiv:1704.01691v2 fatcat:5w6gxzdg45b4piiogt3xv2gmla

Multi-space Variational Encoder-Decoders for Semi-supervised Labeled Sequence Transduction

Chunting Zhou, Graham Neubig
2017 Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)  
The generative model can use neural networks to handle both discrete and continuous latent variables to exploit various features of data.  ...  On the SIGMORPHON morphological inflection benchmark, our model outperforms single-model state-ofart results by a large margin for the majority of languages. 1  ...  Acknowledgments The authors thank Jiatao Gu, Xuezhe Ma, Zihang Dai and Pengcheng Yin for their helpful discussions. This work has been supported in part by an Amazon Academic Research Award.  ... 
doi:10.18653/v1/p17-1029 dblp:conf/acl/ZhouN17 fatcat:6u4b6fex5fflzo4mhlu6j44chq

Language as a Latent Variable: Discrete Generative Models for Sentence Compression [article]

Yishu Miao, Phil Blunsom
2016 arXiv   pre-print
In this work we explore deep generative models of text in which the latent representation of a document is itself drawn from a discrete language model distribution.  ...  We formulate a variational auto-encoder for inference in this model and apply it to the task of compressing sentences.  ...  generative models, a significant contribution of our work is a process for reducing variance for discrete sampling-based variational inference.  ... 
arXiv:1609.07317v2 fatcat:fppddfdenfgkhjaadassg4dxcy

Language as a Latent Variable: Discrete Generative Models for Sentence Compression

Yishu Miao, Phil Blunsom
2016 Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing  
In this work we explore deep generative models of text in which the latent representation of a document is itself drawn from a discrete language model distribution.  ...  We formulate a variational auto-encoder for inference in this model and apply it to the task of compressing sentences.  ...  The most common family of variational autoencoders relies on the reparameterisation trick, which is not applicable for our discrete latent language model.  ... 
doi:10.18653/v1/d16-1031 dblp:conf/emnlp/MiaoB16 fatcat:hdvrqfdv6rhddg3cniw5e7ft7m

Direct Simultaneous Speech-to-Speech Translation with Variational Monotonic Multihead Attention [article]

Xutai Ma, Hongyu Gong, Danni Liu, Ann Lee, Yun Tang, Peng-Jen Chen, Wei-Ning Hsu, Phillip Koehn, Juan Pino
2022 arXiv   pre-print
We also introduce the variational monotonic multihead attention (V-MMA), to handle the challenge of inefficient policy learning in speech simultaneous translation.  ...  an unsupervised manner, are predicted from the model and passed directly to a vocoder for speech synthesis on-the-fly.  ...  A direction of further work can be designing a simultaneous policy specifically for discrete units.  ... 
arXiv:2110.08250v2 fatcat:wcaduwxmc5h2bboh5yeaqqcena

SAM: Semantic Attribute Modulation for Language Modeling and Style Variation [article]

Wenbo Hu, Lifeng Hua, Lei Li, Hang Su, Tian Wang, Ning Chen, Bo Zhang
2017 arXiv   pre-print
This paper presents a Semantic Attribute Modulation (SAM) for language modeling and style variation.  ...  Moreover, we present a style variation for the lyric generation using SAM, which shows a strong connection between the style variation and the semantic attributes.  ...  Then, we use SAM to do language modeling and style variation for language generation.  ... 
arXiv:1707.00117v3 fatcat:vvfzccjiljalxjqkyvsxmequpy

Speech-to-speech Translation between Untranscribed Unknown Languages [article]

Andros Tjandra, Sakriani Sakti, Satoshi Nakamura
2019 arXiv   pre-print
Second, we train a sequence-to-sequence model that directly maps the source language speech to the target language's discrete representation.  ...  Our proposed method consists of two steps: First, we train and generate discrete representation with unsupervised term discovery with a discrete quantized autoencoder.  ...  Therefore, to capture the context without any supervision, we use a generative model named a vector quantized variational autoencoder (VQ-VAE) [15] to extract the discrete symbols.  ... 
arXiv:1910.00795v2 fatcat:hi57dybnjrfbjcbq2ejoick2eq

Deep Bayesian Natural Language Processing

Jen-Tzung Chien
2019 Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: Tutorial Abstracts  
The "distribution function" in discrete or continuous latent variable model for natural language may not be properly decomposed or estimated.  ...  Attention-based models for speech recogni- tion. In Advances in Neural Information Processing Systems, pages 577-585.  ... 
doi:10.18653/v1/p19-4006 dblp:conf/acl/Chien19 fatcat:bj6qf6cpkffz3oxinswh5fy4ry

Weakly-Supervised Speech-to-Text Mapping with Visually Connected Non-Parallel Speech-Text Data Using Cyclic Partially-Aligned Transformer

Johanes Effendi, Sakriani Sakti, Satoshi Nakamura
2021 Conference of the International Speech Communication Association  
Unfortunately, for many other languages, such resources are usually unavailable.  ...  First, we train a Transformer-based vector-quantized variational autoencoder (VQ-VAE) to produce a discrete speech representation in a self-supervised manner.  ...  Then, we trained the VQ-VAE model using the speech data, and generated the code sequence as a discrete speech representation.  ... 
doi:10.21437/interspeech.2021-970 dblp:conf/interspeech/EffendiSN21 fatcat:zukrh6ogwngoxko4jqqtsnbiqi

On Controlled DeEntanglement for Natural Language Processing [article]

SaiKrishna Rallabandi
2019 arXiv   pre-print
I present mathematical analysis from information theory to show that employing stochasticity leads to controlled de-entanglement of relevant factors of variation at various levels.  ...  Unwritten Languages Let us consider building speech technology for unwritten or under-resourced languages.  ...  However, the language which is also present in the utterance can be approximated to be sampled from a discrete prior distribution.  ... 
arXiv:1909.09964v1 fatcat:mi5wm7pnxrddplwluwyqlauuoe

End-to-end Image-to-speech Generation for Untranscribed Unknown Languages

Johanes Effendi, Sakriani Sakti, Satoshi Nakamura
2021 IEEE Access  
We use a self-supervised transformerbased vector-quantized variational autoencoder (VQ-VAE), which has been proven to deliver a promising discretization score for untranscribed unknown languages in the  ...  across languages, especially for an unseen language.  ...  SATOSHI NAKAMURA is Professor of Graduate School of Science and Technology, Nara Institute of Science and Technology, Japan, Project Leader of Tourism Information Analytics Team of RIKEN, Center for  ... 
doi:10.1109/access.2021.3071541 fatcat:7ispiemcgvfzvmoaov7arx3veu
« Previous Showing results 1 — 15 out of 127,895 results