Filters








5,968 Hits in 4.5 sec

Refined α-Divergence Variational Inference via Rejection Sampling [article]

Rahul Sharma, Abhishek Kumar, Piyush Rai
2019 arXiv   pre-print
We present an approximate inference method, based on a synergistic combination of Rényi α-divergence variational inference (RDVI) and rejection sampling (RS).  ...  RDVI is based on minimization of Rényi α-divergence D_α(p||q) between the true distribution p(x) and a variational approximation q(x); RS draws samples from a distribution p(x) = p̃(x)/Z_p using a proposal  ...  Introduction We present an approximate inference method, based on a synergistic combination of Rényi αdivergence variational inference (RDVI) and rejection sampling (RS).  ... 
arXiv:1909.07627v3 fatcat:czubmsuss5ekxiupyvvv4qdwgm

alpha-Deep Probabilistic Inference (alpha-DPI): efficient uncertainty quantification from exoplanet astrometry to black hole feature extraction [article]

He Sun, Katherine L. Bouman, Paul Tiede, Jason J. Wang, Sarah Blunt, Dimitri Mawet
2022 arXiv   pre-print
Traditional approaches for posterior estimation include sampling-based methods and variational inference.  ...  In this paper, we propose alpha-DPI, a deep learning framework that first learns an approximate posterior using alpha-divergence variational inference paired with a generative neural network, and then  ...  α-divergence Variational Inference (α-VI) with Normalizing Flow VI is an approach that solves an optimization problem to estimate a posterior distribution.  ... 
arXiv:2201.08506v2 fatcat:5t6cchz7ffdmrk3tq5aruk7jxi

Automated Variational Inference in Probabilistic Programming [article]

David Wingate, Theophane Weber
2013 arXiv   pre-print
We present a new algorithm for approximate inference in probabilistic programs, based on a stochastic gradient for variational programs.  ...  We show how to automatically derive mean-field probabilistic programs and optimize them, and demonstrate that our perspective improves inference efficiency over other algorithms.  ...  While they do highlight the need for optimizing the parameters of the variational program, they do not offer a general algorithm for doing so, instead suggesting rejection sampling or importance sampling  ... 
arXiv:1301.1299v1 fatcat:jchutrnmkfeqbob27lk5oz3hy4

A Contrastive Divergence for Combining Variational Inference and MCMC [article]

Francisco J. R. Ruiz, Michalis K. Titsias
2019 arXiv   pre-print
To make inference tractable, we introduce the variational contrastive divergence (VCD), a new divergence that replaces the standard Kullback-Leibler (KL) divergence used in VI.  ...  The VCD objective can be optimized efficiently with respect to the variational parameters via stochastic optimization.  ...  Method Description Here we formally describe the method for refining the variational distribution with Markov chain Monte Carlo (MCMC) sampling, as well as the variational contrastive divergence (VCD)  ... 
arXiv:1905.04062v2 fatcat:ofmqcqgrgrepdj2bp553yfd7zu

Parallelizing MCMC via Weierstrass Sampler [article]

Xiangyu Wang, David B. Dunson
2014 arXiv   pre-print
The new sampler approximates the full data posterior samples via combining the posterior draws from independent subset MCMC chains, and thus enjoys a higher computational efficiency.  ...  The algorithm is referred to as a Weierstrass refinement sampler, because samples from an initial rough approximation to the posterior (obtained via Laplace, variational approximations or other methods  ...  Since the model is multidimensional, we drew 2000 samples via the sequential rejection sampling described in Weierstrass (sequential) rejection sampling correctly recognized the posterior modes.  ... 
arXiv:1312.4605v2 fatcat:kay5rt5kgjdoxeacrff4m2fj54

Cooperative Training of Fast Thinking Initializer and Slow Thinking Solver for Conditional Learning [article]

Jianwen Xie, Zilong Zheng, Xiaolin Fang, Song-Chun Zhu, Ying Nian Wu
2021 arXiv   pre-print
We propose to learn the two models jointly, where the fast thinking initializer serves to initialize the sampling of the slow thinking solver, and the solver refines the initial output by an iterative  ...  The solver learns from the difference between the refined output and the observed output, while the initializer learns from how the solver refines its initial output.  ...  Those baselines include GAN-based, flow-based, and variational inference methods.  ... 
arXiv:1902.02812v3 fatcat:yb5aq6rzlvho3mc3p7yfznndja

MCMC-Interactive Variational Inference [article]

Quan Zhang, Huangjie Zheng, Mingyuan Zhou
2020 arXiv   pre-print
Constructing a variational distribution followed by a short Markov chain that has parameters to learn, MIVI takes advantage of the complementary properties of variational inference and MCMC to encourage  ...  On one hand, with the variational distribution locating high posterior density regions, the Markov chain is optimized within the variational inference framework to efficiently target the posterior despite  ...  Variational inference (VI) approximates posterior p(z | x) with variational distribution q(z) by minimizing KL(q(z) || p(z | x)), the Kullback-Leibler (KL) divergence from p(z | x) to q(z) (Jordan et  ... 
arXiv:2010.02029v1 fatcat:kbq3obwkfrdrtj3fyktwe3gdkq

Quantum Inspired Training for Boltzmann Machines [article]

Nathan Wiebe, Ashish Kapoor, Christopher Granade, Krysta M Svore
2015 arXiv   pre-print
We present an efficient classical algorithm for training deep Boltzmann machines (DBMs) that uses rejection sampling in concert with variational approximations to estimate the gradients of the training  ...  Our rejection sampling approach can yield more accurate gradients than low-order contrastive divergence training and the costs incurred in finding increasingly accurate gradients can be easily parallelized  ...  Instrumental Rejection Sampling The main insight behind our result, and stemming from [1] , is that variational approximations to the Gibbs state can be used to make training with rejection sampling much  ... 
arXiv:1507.02642v1 fatcat:wl3lgye55jdyplefpngpewiymu

Advances in Variational Inference [article]

Cheng Zhang, Judith Butepage, Hedvig Kjellstrom, Stephan Mandt
2018 arXiv   pre-print
with atypical divergences, and (d) amortized VI, which implements the inference over local latent variables with inference networks.  ...  In this review, we give an overview of recent trends in variational inference.  ...  For α → 1, we recover standard VI (involving the KL divergence). α-divergences have recently been used in variational inference [103] , [104] .  ... 
arXiv:1711.05597v3 fatcat:st53lmyx5ndpvezmw6vhw4fnhy

Advances in Variational Inference

Cheng Zhang, Judith Butepage, Hedvig Kjellstrom, Stephan Mandt
2018 IEEE Transactions on Pattern Analysis and Machine Intelligence  
with atypical divergences, and (d) amortized VI, which implements the inference over local latent variables with inference networks.  ...  In this review, we give an overview of recent trends in variational inference.  ...  For α → 1, we recover standard VI (involving the KL divergence). α-divergences have recently been used in variational inference [103] , [104] .  ... 
doi:10.1109/tpami.2018.2889774 fatcat:xffyfbw5w5c4dklgs3uvwynp3u

Latent Space Refinement for Deep Generative Models [article]

Ramon Winterhalder, Marco Bellagente, Benjamin Nachman
2021 arXiv   pre-print
We extend this idea to all types of generative models and show how latent space refinement via iterated generative modeling can circumvent topological obstructions and improve precision.  ...  Recent proposals have suggested using classifier weights to refine the learned density of deep generative models.  ...  This can be for instance done directly using the following methods: • Rejection sampling: We can use the weights w(z) to perform rejection sampling on the latent space.  ... 
arXiv:2106.00792v2 fatcat:otqmjkryurhebhmxlpj2kf2wo4

Measuring Uncertainty through Bayesian Learning of Deep Neural Network Structure [article]

Zhijie Deng, Yucen Luo, Jun Zhu, Bo Zhang
2021 arXiv   pre-print
We then develop an efficient stochastic variational inference approach which unifies the learning of both network structure and weights.  ...  This paper investigates a new line of Bayesian deep learning by performing Bayesian inference on network structure.  ...  Similar as the "cold posterior" trick [55] , we propose to sharpen the variational posterior of the structure via two refinements.  ... 
arXiv:1911.09804v3 fatcat:p7tvfrpnpfawhnmcuawp2mh62i

Energy-Inspired Models: Learning with Sampler-Induced Distributions [article]

Dieterich Lawson, George Tucker, Bo Dai, Rajesh Ranganath
2020 arXiv   pre-print
Moreover, EIMs allow us to generalize a recent connection between multi-sample variational lower bounds and auxiliary variable variational inference.  ...  As a result, inference in EBMs relies on approximate sampling algorithms, leading to a mismatch between the model and inference.  ...  As a result, semi-implicit variational inference is simply auxiliary variable variational inference by another name.  ... 
arXiv:1910.14265v2 fatcat:z72wzvq6areh7kw5q4rm56cpsy

Dating Primate Divergences through an Integrated Analysis of Palaeontological and Molecular Data

Richard D. Wilkinson, Michael E. Steiper, Christophe Soligo, Robert D. Martin, Ziheng Yang, Simon Tavaré
2010 Systematic Biology  
Estimation of divergence times is usually done using either the fossil record or sequence data from modern species.  ...  [Approximate Bayesian computation; molecular phylogeny; palaeontological data; primate divergence.]  ...  A rejection-based ABC algorithm for inference would be: Optimal Subtree Selection (OSS) 1. Draw parameters θ = (τ, α, ψ, p, λ) from π(•). 2. Simulate a tree and fossil finds using parameter θ.  ... 
doi:10.1093/sysbio/syq054 pmid:21051775 pmcid:PMC2997628 fatcat:3baaawb64ra4xmd2bq5jfzrf3q

ParaDRAM: A Cross-Language Toolbox for Parallel High-Performance Delayed-Rejection Adaptive Metropolis Markov Chain Monte Carlo Simulations [article]

Amir Shahmoradi, Fatemeh Bagheri
2020 arXiv   pre-print
We present ParaDRAM, a high-performance Parallel Delayed-Rejection Adaptive Metropolis Markov Chain Monte Carlo software for optimization, sampling, and integration of mathematical objective functions  ...  encountered in scientific inference.  ...  The numerical computation of the IAC, however, poses another challenge to decorrelating MCMC samples since the variance of its estimator in (18) diverges to infinity.  ... 
arXiv:2008.09589v1 fatcat:kjxkj63syjb67it4bdm6f5q6rq
« Previous Showing results 1 — 15 out of 5,968 results