A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Filters
Discrete Variational Attention Models for Language Generation
[article]
2021
arXiv
pre-print
Preserved Fulltext
Other Versions
To tackle these issues, in this paper we propose the discrete variational attention model with categorical distribution over the attention mechanism owing to the discrete nature in languages. ...
Variational autoencoders have been widely applied for natural language generation, however, there are two long-standing problems: information under-representation and posterior collapse. ...
Conclusion In this paper, we propose discrete variational attention model, a new algorithm for natural language generation. ...
arXiv:2004.09764v4
fatcat:4ipk5qpxynhzxer4nbatkfhhr4
Discrete Auto-regressive Variational Attention Models for Text Modeling
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
We further design discrete latent space for the variational attention and mathematically show that our model is free from posterior collapse. ...
In this paper, we propose Discrete Auto-regressive Variational Attention Model (DAVAM) to address the challenges. ...
CONCLUSION In this paper, we propose the discrete auto-regressive variational attention model, a new deep generative model for text modeling. ...
arXiv:2106.08571v1
fatcat:hindfinnnbho3ixpoaiz4aqfi4
Variational Attention Using Articulatory Priors for Generating Code Mixed Speech Using Monolingual Corpora
2019
Interspeech 2019
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Subjective evaluation shows that our systems are capable of generating high quality synthesis in code mixed scenarios. ...
We subject the prior distribution for such latent variables to match articulatory constraints. ...
Acknowledgements We thank AFRL for funding this research, student volunteers for taking part in the listening evaluations and reviewers for such valuable feedback. ...
doi:10.21437/interspeech.2019-1103
dblp:conf/interspeech/RallabandiB19
fatcat:td4tbtiy6fab3h62lzilzav6be
Morphological Inflection Generation with Multi-space Variational Encoder-Decoders
2017
Proceedings of the CoNLL SIGMORPHON 2017 Shared Task: Universal Morphological Reinflection
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
The system is based on the multi-space variational encoder-decoder (MSVED) method of Zhou and Neubig (2017), which employs both continuous and discrete latent variables for the variational encoder-decoder ...
We discuss some language-specific errors and present result analysis. ...
We thank Matthew Honnibal for pointing out that the data distribution of Wikipedia corpus might be biased. ...
doi:10.18653/v1/k17-2005
dblp:conf/conll/ZhouN17
fatcat:4qdlr6iearbnpenlcedqhuojj4
Multi-space Variational Encoder-Decoders for Semi-supervised Labeled Sequence Transduction
[article]
2017
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
The generative model can use neural networks to handle both discrete and continuous latent variables to exploit various features of data. ...
On the SIGMORPHON morphological inflection benchmark, our model outperforms single-model state-of-art results by a large margin for the majority of languages. ...
Acknowledgments The authors thank Jiatao Gu, Xuezhe Ma, Zihang Dai and Pengcheng Yin for their helpful discussions. This work has been supported in part by an Amazon Academic Research Award. ...
arXiv:1704.01691v2
fatcat:5w6gxzdg45b4piiogt3xv2gmla
Multi-space Variational Encoder-Decoders for Semi-supervised Labeled Sequence Transduction
2017
Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
The generative model can use neural networks to handle both discrete and continuous latent variables to exploit various features of data. ...
On the SIGMORPHON morphological inflection benchmark, our model outperforms single-model state-ofart results by a large margin for the majority of languages. 1 ...
Acknowledgments The authors thank Jiatao Gu, Xuezhe Ma, Zihang Dai and Pengcheng Yin for their helpful discussions. This work has been supported in part by an Amazon Academic Research Award. ...
doi:10.18653/v1/p17-1029
dblp:conf/acl/ZhouN17
fatcat:6u4b6fex5fflzo4mhlu6j44chq
Language as a Latent Variable: Discrete Generative Models for Sentence Compression
[article]
2016
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
In this work we explore deep generative models of text in which the latent representation of a document is itself drawn from a discrete language model distribution. ...
We formulate a variational auto-encoder for inference in this model and apply it to the task of compressing sentences. ...
generative models, a significant contribution of our work is a process for reducing variance for discrete sampling-based variational inference. ...
arXiv:1609.07317v2
fatcat:fppddfdenfgkhjaadassg4dxcy
Language as a Latent Variable: Discrete Generative Models for Sentence Compression
2016
Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
In this work we explore deep generative models of text in which the latent representation of a document is itself drawn from a discrete language model distribution. ...
We formulate a variational auto-encoder for inference in this model and apply it to the task of compressing sentences. ...
The most common family of variational autoencoders relies on the reparameterisation trick, which is not applicable for our discrete latent language model. ...
doi:10.18653/v1/d16-1031
dblp:conf/emnlp/MiaoB16
fatcat:hdvrqfdv6rhddg3cniw5e7ft7m
Direct Simultaneous Speech-to-Speech Translation with Variational Monotonic Multihead Attention
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Note that this fulltext copy is not of the "primary" version of this work. The version it corresponds to is:
2021 pre-print arXiv:2110.08250v1 fatcat:ipqvrnvrdzbvnbdbr6uau442ey
We also introduce the variational monotonic multihead attention (V-MMA), to handle the challenge of inefficient policy learning in speech simultaneous translation. ...
an unsupervised manner, are predicted from the model and passed directly to a vocoder for speech synthesis on-the-fly. ...
A direction of further work can be designing a simultaneous policy specifically for discrete units. ...
arXiv:2110.08250v2
fatcat:wcaduwxmc5h2bboh5yeaqqcena
SAM: Semantic Attribute Modulation for Language Modeling and Style Variation
[article]
2017
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
This paper presents a Semantic Attribute Modulation (SAM) for language modeling and style variation. ...
Moreover, we present a style variation for the lyric generation using SAM, which shows a strong connection between the style variation and the semantic attributes. ...
Then, we use SAM to do language modeling and style variation for language generation. ...
arXiv:1707.00117v3
fatcat:vvfzccjiljalxjqkyvsxmequpy
Speech-to-speech Translation between Untranscribed Unknown Languages
[article]
2019
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Second, we train a sequence-to-sequence model that directly maps the source language speech to the target language's discrete representation. ...
Our proposed method consists of two steps: First, we train and generate discrete representation with unsupervised term discovery with a discrete quantized autoencoder. ...
Therefore, to capture the context without any supervision, we use a generative model named a vector quantized variational autoencoder (VQ-VAE) [15] to extract the discrete symbols. ...
arXiv:1910.00795v2
fatcat:hi57dybnjrfbjcbq2ejoick2eq
Deep Bayesian Natural Language Processing
2019
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: Tutorial Abstracts
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
The "distribution function" in discrete or continuous latent variable model for natural language may not be properly decomposed or estimated. ...
Attention-based models for speech recogni-
tion. In Advances in Neural Information Processing
Systems, pages 577-585. ...
doi:10.18653/v1/p19-4006
dblp:conf/acl/Chien19
fatcat:bj6qf6cpkffz3oxinswh5fy4ry
Weakly-Supervised Speech-to-Text Mapping with Visually Connected Non-Parallel Speech-Text Data Using Cyclic Partially-Aligned Transformer
2021
Conference of the International Speech Communication Association
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Unfortunately, for many other languages, such resources are usually unavailable. ...
First, we train a Transformer-based vector-quantized variational autoencoder (VQ-VAE) to produce a discrete speech representation in a self-supervised manner. ...
Then, we trained the VQ-VAE model using the speech data, and generated the code sequence as a discrete speech representation. ...
doi:10.21437/interspeech.2021-970
dblp:conf/interspeech/EffendiSN21
fatcat:zukrh6ogwngoxko4jqqtsnbiqi
On Controlled DeEntanglement for Natural Language Processing
[article]
2019
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
I present mathematical analysis from information theory to show that employing stochasticity leads to controlled de-entanglement of relevant factors of variation at various levels. ...
Unwritten Languages Let us consider building speech technology for unwritten or under-resourced languages. ...
However, the language which is also present in the utterance can be approximated to be sampled from a discrete prior distribution. ...
arXiv:1909.09964v1
fatcat:mi5wm7pnxrddplwluwyqlauuoe
End-to-end Image-to-speech Generation for Untranscribed Unknown Languages
2021
IEEE Access
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
We use a self-supervised transformerbased vector-quantized variational autoencoder (VQ-VAE), which has been proven to deliver a promising discretization score for untranscribed unknown languages in the ...
across languages, especially for an unseen language. ...
SATOSHI NAKAMURA is Professor of Graduate School of Science and Technology, Nara Institute of Science and Technology, Japan, Project Leader of Tourism Information Analytics Team of RIKEN, Center for ...
doi:10.1109/access.2021.3071541
fatcat:7ispiemcgvfzvmoaov7arx3veu
« Previous
Showing results 1 — 15 out of 127,895 results