Filters








201,969 Hits in 4.5 sec

Bounding the Test Log-Likelihood of Generative Models [article]

Yoshua Bengio, Li Yao, Kyunghyun Cho
2014 arXiv   pre-print
We revisit this idea, propose a more efficient estimator, and prove that it provides a lower bound on the true test log-likelihood, and an unbiased estimator as the number of generated samples goes to  ...  A previously proposed method is based on constructing a non-parametric density estimator of the model's probability function from samples generated by the model.  ...  Acknowledgements We would like to thank the developers of Theano (Bergstra et al., 2010; Bastien et al., 2012) , as well NSERC, CIFAR, Compute Canada, and Calcul Québec for funding.  ... 
arXiv:1311.6184v4 fatcat:ckj7nnhghvcybiownf5g6ubwfi

Joint Multimodal Learning with Deep Generative Models [article]

Masahiro Suzuki, Kotaro Nakayama, Yutaka Matsuo
2016 arXiv   pre-print
In other words, it models a joint distribution of modalities.  ...  Furthermore, to be able to generate missing modalities from the remaining modalities properly, we develop an additional method, JMVAE-kl, that is trained by reducing the divergence between JMVAE's encoder  ...  We can calculate the lower bounds of these log-likelihoods as follows: Table 3 : 3 Evaluation of test log-likelihood.  ... 
arXiv:1611.01891v1 fatcat:db2uiveberb4rko5jrm5ngyibq

On the Quantitative Analysis of Decoder-Based Generative Models [article]

Yuhuai Wu, Yuri Burda, Ruslan Salakhutdinov, Roger Grosse
2017 arXiv   pre-print
Using this technique, we analyze the performance of decoder-based models, the effectiveness of existing log-likelihood estimators, the degree of overfitting, and the degree to which these models miss important  ...  Unfortunately, it can be difficult to quantify the performance of these models because of the intractability of log-likelihood estimation, and inspecting samples can be misleading.  ...  According to the log-likelihood evaluation, we find digit "2" is the hardest digit for modelling. In this section we investigate the quality of modelling "2" of each model.  ... 
arXiv:1611.04273v2 fatcat:a4jas465qbdjpbqazgzmchcenq

Provably robust deep generative models [article]

Filipe Condessa, Zico Kolter
2020 arXiv   pre-print
To do so, we first formally define a (certifiably) robust lower bound on the variational lower bound of the likelihood, and then show how this bound can be optimized during training to produce a robust  ...  as to drastically lower their likelihood under the model).  ...  lower bound of the likelihood and optimizing this bound to train a robust generative model (proVAE).  ... 
arXiv:2004.10608v1 fatcat:frllvivoubfehmyxlzeq5ifxi4

Likelihood Dominance Spatial Inference

R. Kelley Pace, James P. LeSage
2003 Geographical Analysis  
Of course, this also translates into a lower bound on the deviance, which equals twice the difference between the profile log-likelihood of the overall model and the restricted model.  ...  Our approach maps these bounds on the autoregressive parameter to a lower bound on the likelihood ratio associated with testing hypotheses on regression parameters.  ...  We employ a more general model that subsumes these two models and thus permits testing of the heterogeneity versus spatial dependence hypotheses.  ... 
doi:10.1111/j.1538-4632.2003.tb01105.x fatcat:6xdwpxk5fvevhn5ubdralcr7ha

Ladder Variational Autoencoders [article]

Casper Kaae Sønderby, Tapani Raiko, Lars Maaløe, Søren Kaae Sønderby, Ole Winther
2016 arXiv   pre-print
We show that this model provides state of the art predictive log-likelihood and tighter log-likelihood lower bound compared to the purely bottom-up inference in layered Variational Autoencoders and other  ...  generative models.  ...  Acknowledgments This research was supported by the Novo Nordisk Foundation, Danish Innovation Foundation and the NVIDIA Corporation with the donation of TITAN X and Tesla K40 GPUs.  ... 
arXiv:1602.02282v3 fatcat:53z5qnfysbfcpmo6stklz7vwfy

Layer-wise learning of deep generative models [article]

Ludovic Arnold, Yann Ollivier
2013 arXiv   pre-print
We interpret auto-encoders in this setting as generative models, by showing that they train a lower bound of this criterion.  ...  We test the new learning procedure against a state of the art method (stacked RBMs), and find it to improve performance.  ...  Thus, the BLM upper bound could be used as a test for the opportunity of adding layers to a model. Training BLM upper bound vs validation log-likelihood.  ... 
arXiv:1212.1524v2 fatcat:ud5wds5farh4tg35ks7y6kox7a

Information Theoretic Lower Bounds on Negative Log Likelihood [article]

Luis A. Lastras
2019 arXiv   pre-print
One way to address this question is to find an upper bound on the probability (equivalently a lower bound on the negative log likelihood) that the model can assign to some data as one varies the prior  ...  Moreover, we will show that if changing the prior can improve the log likelihood, then there is a way to change the likelihood function instead and attain the same log likelihood, and thus rate-distortion  ...  This figure is a graphical depiction of the upper and lower bounds (3) and (4) , applied to the test data set negative log likelihood.  ... 
arXiv:1904.06395v1 fatcat:wsxfe2dobvglph4txy62s5hu3q

Entropic GANs meet VAEs: A Statistical Approach to Compute Sample Likelihoods in GANs [article]

Yogesh Balaji, Hamed Hassani, Rama Chellappa, Soheil Feizi
2019 arXiv   pre-print
VAEs consider an explicit probability model for the data and compute a generative distribution by maximizing a variational lower-bound on the log-likelihood function.  ...  The lack of having explicit probability models in GANs prohibits computation of sample likelihoods in their frameworks and limits their use in statistical inference problems.  ...  We can lower-bound the log likelihood of this sample as log f Y (y test ) log likelihood ≥ − E P * X Y =y test (y test , G * (x)) distance to the generative model + H P * X Y =y test coupling entropy +  ... 
arXiv:1810.04147v2 fatcat:heewvqx2hncbtbf6faaknar7hm

Learning in Variational Autoencoders with Kullback-Leibler and Renyi Integral Bounds [article]

Septimia Sârbu and Riccardo Volpi and Alexandra Peşte and Luigi Malagò
2018 arXiv   pre-print
In this paper we propose two novel bounds for the log-likelihood based on Kullback-Leibler and the Rényi divergences, which can be used for variational inference and in particular for the training of Variational  ...  often appear unless noise is added, either to the dataset during training or to the generative model given by the decoder.  ...  Acknowledgements We would like to thank Andrei Ciuparu for useful discussions related to the training issues we encountered with the original ELBO and an anonymous reviewer for suggesting the reference  ... 
arXiv:1807.01889v1 fatcat:3vupo7k25fgabiqf6h2p2gnnfa

Semi-supervised learning objectives as log-likelihoods in a generative model of data curation [article]

Stoil Ganev, Laurence Aitchison
2021 arXiv   pre-print
Here, we note that benchmark image datasets such as CIFAR-10 are carefully curated, and we formulate SSL objectives as a log-likelihood in a generative model of data curation that was initially developed  ...  SSL methods, from entropy minimization and pseudo-labelling, to state-of-the-art techniques similar to FixMatch can be understood as lower-bounds on our principled log-likelihood.  ...  CONCLUSION We showed that low-density separation SSL objectives can be understood as a lower-bound on a log-probability which arises from a principled generative model of data curation.  ... 
arXiv:2008.05913v2 fatcat:xc3tqmyrpnbcpbs7hvdeiq55gm

Importance Weighted Autoencoders [article]

Yuri Burda, Roger Grosse, Ruslan Salakhutdinov
2016 arXiv   pre-print
We present the importance weighted autoencoder (IWAE), a generative model with the same architecture as the VAE, but which uses a strictly tighter log-likelihood lower bound derived from importance weighting  ...  We show empirically that IWAEs learn richer latent space representations than VAEs, leading to improved test log-likelihood on density estimation benchmarks.  ...  ACKNOWLEDGEMENTS This research was supported by NSERC, the Fields Institute, and Samsung.  ... 
arXiv:1509.00519v4 fatcat:ovec4vplv5ednapqdahfbsbmui

Learning to Generate Samples from Noise through Infusion Training [article]

Florian Bordes, Sina Honari, Pascal Vincent
2017 arXiv   pre-print
The novel training procedure to learn this progressive denoising operation involves sampling from a slightly different chain than the model chain used for generation in the absence of a denoising target  ...  In this work, we investigate a novel training procedure to learn a generative model as the transition operator of a Markov chain, such that, when applied repeatedly on an unstructured random noise sample  ...  ACKNOWLEDGMENTS We would like to thank the developers of Theano (Theano Development Team, 2016) for making this library available to build on, Compute Canada and Nvidia for their computation resources,  ... 
arXiv:1703.06975v1 fatcat:dxgtcc2qcngbtazrgpdy2i7eqq

Accurate and Conservative Estimates of MRF Log-likelihood using Reverse Annealing [article]

Yuri Burda and Roger B. Grosse and Ruslan Salakhutdinov
2014 arXiv   pre-print
We present the Reverse AIS Estimator (RAISE), a stochastic lower bound on the log-likelihood of an approximation to the original MRF model.  ...  Markov random fields (MRFs) are difficult to evaluate as generative models because computing the test log-probabilities requires the intractable partition function.  ...  As discussed in Section 3.1, RAISE is a stochastic lower bound on the log-likelihood of the annealing model p ann , but not necessarily of the RBM itself.  ... 
arXiv:1412.8566v1 fatcat:hfilujbbvzhppoqcu36iuf6yum

Bounded Gaussian process regression

Bjorn Sand Jensen, Jens Brehm Nielsen, Jan Larsen
2013 2013 IEEE International Workshop on Machine Learning for Signal Processing (MLSP)  
We extend the Gaussian process (GP) framework for bounded regression by introducing two bounded likelihood functions that model the noise on the dependent variable explicitly.  ...  We approximate the intractable posterior distributions by the Laplace approximation and expectation propagation and show the properties of the models on an artificial example.  ...  We show that, as expected, the model with the correct noise assumption provides the best expected predictive negative log likelihood (or, alternatively, generalization error).  ... 
doi:10.1109/mlsp.2013.6661916 dblp:conf/mlsp/JensenNL13 fatcat:vknx35vg5vdunftfb4mkorp77m
« Previous Showing results 1 — 15 out of 201,969 results