Filters








1,250 Hits in 4.2 sec

Principles of Bayesian Inference Using General Divergence Criteria

Jack Jewson, Jim Smith, Chris Holmes
2018 Entropy  
for the parameter value minimising the Kullback-Leibler (KL)-divergence between the model and this process (Walker, 2013).  ...  However, it has long been known that minimising the KL-divergence places a large weight on correctly capturing the tails of the sample distribution.  ...  The authors would also like to thank Jeremias Knoblauch for his helpful discussions when revising the article.  ... 
doi:10.3390/e20060442 pmid:33265532 fatcat:4574kuelwnefnd6ek3sc2lruza

Variational Bayes approach for model aggregation in unsupervised classification with Markovian dependency [article]

Stevenn Volant, Marie-Laure Martin Magniette, Stéphane Robin
2011 arXiv   pre-print
The aim is then the estimation of the weights and the posterior probability for one specific model.  ...  The inference is done within a variational Bayesian framework and our aim is to infer the posterior probability of belonging to the class of interest.  ...  Accuracy of the posterior probabilities Once the weights have been estimated, the averaged estimates of the posterior probabilities T t are computed for each approach.  ... 
arXiv:1105.0760v1 fatcat:77aewnexvze6zkne2zrnoxpcam

Preventing Posterior Collapse with Levenshtein Variational Autoencoder [article]

Serhii Havrylov, Ivan Titov
2020 arXiv   pre-print
Intuitively, it corresponds to generating a sequence from the autoencoder and encouraging the model to predict an optimal continuation according to the Levenshtein distance (LD) with the reference sentence  ...  We motivate the method from the probabilistic perspective by showing that it is closely related to optimizing a bound on the intractable Kullback-Leibler divergence of an LD-based kernel density estimator  ...  Directly minimising this objective function is problematic due to the absence of the closed-form solution for the integral, which is caused by RNN parameterisation of each conditional distribution.  ... 
arXiv:2004.14758v1 fatcat:lmh4n2okujd7pogyxo2nhqq6i4

Variational Bayes approach for model aggregation in unsupervised classification with Markovian dependency

Stevenn Volant, Marie-Laure Martin Magniette, Stéphane Robin
2012 Computational Statistics & Data Analysis  
An aim is then the estimation of the weights and the posterior probability for a specific model.  ...  The inference is performed within a variational Bayesian framework and the aim is to infer the posterior probability of belonging to the class of interest.  ...  Acknowledgments The authors thank the reviewers for their helpful comments which have enabled us to improve the presentation of our work.  ... 
doi:10.1016/j.csda.2012.01.027 fatcat:gkfo5dsqgnce5fr4uarmnxsob4

Sequential, Bayesian Geostatistics: A Principled Method for Large Data Sets

Dan Cornford, Lehel Csato, Manfred Opper
2005 Geographical Analysis  
In this paper, we present a Bayesian method for estimating the posterior mean and covariance structures of a Gaussian random field using a sequential estimation algorithm.  ...  By imposing sparsity in a well-defined framework, the algorithm retains a subset of 'basis vectors' which 1 best represent the 'true' posterior Gaussian random field model in the relative entropy sense  ...  We are grateful to the conference organisers for inviting submission of this paper, and to the reviewers for their helpful comments and suggestions which have greatly improved the readability of the paper  ... 
doi:10.1111/j.1538-4632.2005.00635.x fatcat:fc3o6nhj5nfqnlghgdaw4kkt4q

Bernstein - von Mises theorem and misspecified models: a review [article]

Natalia Bochkina
2022 arXiv   pre-print
In particular we focus on consistency, i.e. convergence of the posterior distribution to the point mass at the best parametric approximation to the true model, and conditions for it to be locally Gaussian  ...  For well specified regular models, variance of the Gaussian approximation coincides with the Fisher information, making Bayesian inference asymptotically efficient.  ...  This review was in part motivated by the discussion of the author with Peter Grünwald, Pierre Jacob and Jeffrey Miller during a Research in Groups meeting sponsored by the International Centre for Mathematical  ... 
arXiv:2204.13614v1 fatcat:eoqal5zfwndppesinsketmr53u

Online Bayesian Inference for the Parameters of PRISM Programs [chapter]

James Cussens
2012 Lecture Notes in Computer Science  
This paper presents a method for approximating posterior distributions over the parameters of a given PRISM program.  ...  An approximation is effected by merging products of Dirichlet distributions. An analysis of the quality of the approximation is presented.  ...  Acknowledgements Thanks to the anonymous reviewers for their comments and suggestions.  ... 
doi:10.1007/978-3-642-31951-8_4 fatcat:osie7hmf5rdndbtj24ztfldwya

Multi-Modal Mean-Fields via Cardinality-Based Clamping [article]

Pierre Baqué, François Fleuret, Pascal Fua
2016 arXiv   pre-print
However, since it models the posterior probability distribution as a product of marginal probabilities, it may fail to properly account for important dependencies between variables.  ...  We therefore replace the fully factorized distribution of Mean Field by a weighted mixture of such distributions, that similarly minimizes the KL-Divergence to the true posterior.  ...  It then estimates the probabilities of occupancy at every discrete location as the marginals of a product law minimizing the KL divergence from the "true" conditional posterior distribution, formulated  ... 
arXiv:1611.07941v1 fatcat:gm2zvhdtjnh2djfset2wz3dvh4

Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models [article]

Sam Bond-Taylor, Adam Leach, Yang Long, Chris G. Willcocks
2021 arXiv   pre-print
Deep generative modelling is a class of techniques that train deep neural networks to model the distribution of training samples.  ...  These techniques are drawn under a single cohesive framework, comparing and contrasting to explain the premises behind each, while reviewing current state-of-the-art advances and implementations.  ...  Alternatively, quantile regression, which minimises Wasserstein distance, can be used to learn an approximation of the inverse cumulative distribution [173] .  ... 
arXiv:2103.04922v2 fatcat:nivlg3whyjhadhwdl2tsh5yciy

Relabelling Algorithms for Large Dataset Mixture Models [article]

Wanchuang Zhu, Yanan Fan
2014 arXiv   pre-print
Label switching arises because the posterior is invariant to permutations of the component parameters.  ...  Bayesian estimation of such models typically relies on sampling from the posterior distribution using Markov chain Monte Carlo.  ...  the class probabilities to a fixed reference label, Yao (2012) proposes to assign the probabilities for each possible labels by fitting a mixture model to the permutation symmetric posterior.  ... 
arXiv:1403.2137v1 fatcat:gionnyflovaz3f4uxwqocavwia

Dynamic Trees: A Structured Variational Method Giving Efficient Propagation Rules [article]

Amos J. Storkey
2013 arXiv   pre-print
The marginals calculated give better approximations to the posterior than loopy propagation on a small toy problem.  ...  For this reason approximate methods such as sampling or mean field approaches have been used. However, mean field approximations assume a factorized distribution over node states.  ...  Acknowledgements The work of Amos Starkey is supported through EPSRC grant GR/L78161 Probabilistic Models for Sequences. The author thanks Chris Williams for many helpful comments.  ... 
arXiv:1301.3895v1 fatcat:nlvvoxyyn5bpvjylldauu7c3n4

Online Bayesian inference for the parameters of PRISM programs

James Cussens
2012 Machine Learning  
This paper presents a method for approximating posterior distributions over the parameters of a given PRISM program.  ...  An approximation is effected by merging products of Dirichlet distributions. An analysis of the quality of the approximation is presented.  ...  Acknowledgements Thanks to the anonymous reviewers for their comments and suggestions.  ... 
doi:10.1007/s10994-012-5305-8 fatcat:e5dpy5imvfal7i6ko3jtmzpfki

Bayesian Distributional Policy Gradients [article]

Luchen Li, A. Aldo Faisal
2021 arXiv   pre-print
Distributional Reinforcement Learning (RL) maintains the entire probability distribution of the reward-to-go, i.e. the return, providing more learning signals that account for the uncertainty associated  ...  The proposed algorithm, BDPG (Bayesian Distributional Policy Gradients), uses adversarial training in joint-contrastive learning to estimate a variational posterior from the returns.  ...  Acknowledgments We are grateful for our funding support: a Department of Computing PhD Award to LL and a UKRI Turing AI Fellowship (EP/V025449/1) to AAF.  ... 
arXiv:2103.11265v2 fatcat:f5ykig2iqvbixblazqbsamo3nu

Sophisticated Inference [article]

Karl Friston, Lancelot Da Costa, Danijar Hafner, Casper Hesp, Thomas Parr
2020 arXiv   pre-print
We consider agents with beliefs about the counterfactual consequences of action for states of affairs and beliefs about those latent states.  ...  The recursive form of the free energy functional effectively implements a deep tree search over actions and outcomes in the future.  ...  Disclosure statement The authors have no disclosures or conflict of interest.  ... 
arXiv:2006.04120v1 fatcat:r2buizx5b5cl3h564glpmj3xm4

Fast Approximate Inference in Hybrid Bayesian Networks Using Dynamic Discretisation [chapter]

Helge Langseth, David Marquez, Martin Neil
2013 Lecture Notes in Computer Science  
The most efficient discretisation procedure in terms of cost of inference is known as dynamic discretisation, and was published by Kozlov and Koller in the late 90's.  ...  We consider the mathematical properties of Neil et al.'s algorithm, and challenge it by constructing models that are particularly difficult for that method.  ...  pdf as possible in terms of the KL distance [4] .  ... 
doi:10.1007/978-3-642-38637-4_23 fatcat:3gm6ts55g5abdotckpv4liiwa4
« Previous Showing results 1 — 15 out of 1,250 results