Filters








170,184 Hits in 3.0 sec

Structured Stochastic Variational Inference [article]

Matthew D. Hoffman, David M. Blei
2014 arXiv   pre-print
Stochastic variational inference makes it possible to approximate posterior distributions induced by large datasets quickly using stochastic optimization.  ...  The algorithm relies on the use of fully factorized variational distributions.  ...  Structured Stochastic Variational Inference In this section, we will present two SSVI algorithms.  ... 
arXiv:1404.4114v3 fatcat:hsineim6lbcmriqce4wbexshfq

Beta Process Non-negative Matrix Factorization with Stochastic Structured Mean-Field Variational Inference [article]

Dawen Liang, Matthew D. Hoffman
2014 arXiv   pre-print
We leverage the recently developed stochastic structured mean-field variational inference to relax the conjugacy constraint and restore the dependencies among the latent variables in the approximating  ...  In this paper, we derive a structured mean-field variational inference algorithm for a beta process non-negative matrix factorization (NMF) model with Poisson likelihood.  ...  Stochastic structured mean-field variational inference Following the stochastic structure mean-field variational inference framework, we divide the latent random variables into local: {Z t , s t } T t=  ... 
arXiv:1411.1804v2 fatcat:44fhuvvrn5hyhkscf2qfogzute

Stochastic Collapsed Variational Inference for Structured Gaussian Process Regression Network [article]

Rui Meng, Herbie Lee, Kristofer Bouchard
2021 arXiv   pre-print
This paper presents an efficient variational inference framework for deriving a family of structured gaussian process regression network (SGPRN) models.  ...  Then we propose structured variable distributions and marginalize latent variables, which enables the decomposability of a tractable variational lower bound and leads to stochastic optimization.  ...  stochastic variational inference.  ... 
arXiv:2106.00719v2 fatcat:xjzxarth3bdsvac6vvlarsl4na

Automated Variational Inference in Probabilistic Programming [article]

David Wingate, Theophane Weber
2013 arXiv   pre-print
We present a new algorithm for approximate inference in probabilistic programs, based on a stochastic gradient for variational programs.  ...  This method is efficient without restrictions on the probabilistic program; it is particularly practical for distributions which are not analytically tractable, including highly structured distributions  ...  Finally, stochastic gradient methods are also used in online variational inference algorithms, in particular in the work of Blei et al. in stochastic variational inference (for instance, online LDA [19  ... 
arXiv:1301.1299v1 fatcat:jchutrnmkfeqbob27lk5oz3hy4

Scaling Factorial Hidden Markov Models: Stochastic Variational Inference without Messages [article]

Yin Cheng Ng, Pawel Chilinski, Ricardo Silva
2016 arXiv   pre-print
We propose a scalable inference and learning algorithm for FHMMs that draws on ideas from the stochastic variational inference, neural network and copula literatures.  ...  Our experiments corroborate that the proposed algorithm does not introduce further approximation bias compared to the proven structured mean-field algorithm, and achieves better performance with long sequences  ...  The Message Free Stochastic Variational Inference Algorithm While structured mean-field variational inference and its associated EM algorithms are effective tools for inference and learning in FHMMs with  ... 
arXiv:1608.03817v3 fatcat:fzss76cwwzhanff45khahy3kq4

Stochastic Variational Inference [article]

Matt Hoffman, David M. Blei, Chong Wang, John Paisley
2013 arXiv   pre-print
Stochastic variational inference lets us apply complex Bayesian models to massive data sets.  ...  We develop stochastic variational inference, a scalable algorithm for approximating posterior distributions.  ...  In this appendix, we show how to do stochastic variational inference under the weaker assumption that we can break the global parameter vector β into a set of K subvectors β 1:K such that each conditional  ... 
arXiv:1206.7051v3 fatcat:hsdtdfvs6ne2dnhefqokmtfua4

Variational inference for neural network matrix factorization and its application to stochastic blockmodeling [article]

Onno Kampman, Creighton Heaukulani
2019 arXiv   pre-print
We describe a variational inference algorithm for a neural network matrix factorization model with nonparametric block structure and evaluate its performance on the NIPS co-authorship data set.  ...  Such a probabilistic approach is required, however, when considering the important class of stochastic block models.  ...  Stochastic variational inference We consider letting f θ be a Bayesian neural network and elect a mean-field variational approach to inference.  ... 
arXiv:1905.04502v3 fatcat:sihppqsaubgd5j62prty5vun7m

Graph Representation Learning via Ladder Gamma Variational Autoencoders

Arindam Sarkar, Nikhil Mehta, Piyush Rai
2020 PROCEEDINGS OF THE THIRTIETH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE AND THE TWENTY-EIGHTH INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE  
We present a probabilistic framework for community discovery and link prediction for graph-structured data, based on a novel, gamma ladder variational autoencoder (VAE) architecture.  ...  In addition to leveraging the representational power of multiple layers of stochastic variables via the ladder VAE architecture, our framework offers the following benefits: (1) Unlike existing ladder  ...  Thus we use stochastic gradient variational Bayes (SGVB) (Kingma and Welling 2013; 2014) to perform inference for our model.  ... 
doi:10.1609/aaai.v34i04.6013 fatcat:be2y5orhzbhntd5oeckulc5wtm

Stochastic Blockmodels meet Graph Neural Networks [article]

Nikhil Mehta, Lawrence Carin, Piyush Rai
2019 arXiv   pre-print
They have proven to be successful for various tasks, such as discovering the community structure and link prediction on graph-structured data.  ...  Moreover, our framework is accompanied by a fast recognition model that enables fast inference of the node embeddings (which are of independent interest for inference in SBM and its variants).  ...  Structured Mean-Field: Since the vanilla mean-field variational inference ignores the posterior dependence among the latent variables, we also considered Structured Stochastic Variational Inference (SSVI  ... 
arXiv:1905.05738v1 fatcat:udjb3hyrx5d3bms5kuhan3zkny

Copula variational inference [article]

Dustin Tran, David M. Blei, Edoardo M. Airoldi
2015 arXiv   pre-print
With stochastic optimization, inference on the augmented distribution is scalable.  ...  We develop a general variational inference method that preserves dependency among the latent variables.  ...  Combined with stochastic optimization, variational inference can scale complex statistical models to massive data sets [9, 23, 24] .  ... 
arXiv:1506.03159v2 fatcat:escqrubahvas5h4asobl2jaypq

Truncation-free Online Variational Inference for Bayesian Nonparametric Models

Chong Wang, David M. Blei
2012 Neural Information Processing Systems  
Our method performs better than previous stochastic variational inference algorithms.  ...  We present a truncation-free stochastic variational inference algorithm for Bayesian nonparametric models.  ...  Thus we need to truncate the variational distribution [13, 14] . Truncation is necessary in variational inference because of the mathematical structure of BNP models.  ... 
dblp:conf/nips/WangB12 fatcat:o5z74ffkwngwdb7xbjaggc6gni

Variational Training for Large-Scale Noisy-OR Bayesian Networks

Geng Ji, Dehua Cheng, Huazhong Ning, Changhe Yuan, Hanning Zhou, Liang Xiong, Erik B. Sudderth
2019 Conference on Uncertainty in Artificial Intelligence  
We propose a stochastic variational inference algorithm for training large-scale Bayesian networks, where noisy-OR conditional distributions are used to capture higher-order relationships.  ...  Using stochastic gradient updates based on our variational bounds, we learn noisy-OR Bayesian networks orders of magnitude faster than was possible with prior Monte Carlo learning algorithms, and provide  ...  Michael Hughes and Jialu Liu provided helpful information about baseline topic models and inference algorithms.  ... 
dblp:conf/uai/0001CNYZXS19 fatcat:ibqkwflconhsfjjx3pxi67266m

Stochastic Bayesian Neural Networks [article]

Abhinav Sagar
2021 arXiv   pre-print
Our work builds on variational inference techniques for bayesian neural networks using the original Evidence Lower Bound.  ...  Bayesian neural networks perform variational inference over the weights however calculation of the posterior distribution remains a challenge.  ...  We proposed a new lower bound using variational inference techniques which we named as Stochastic Evidence Lower Bound.  ... 
arXiv:2008.07587v3 fatcat:qtsfg65rg5eqle2sga2jktg3ye

Ladder Variational Autoencoders [article]

Casper Kaae Sønderby, Tapani Raiko, Lars Maaløe, Søren Kaae Sønderby, Ole Winther
2016 arXiv   pre-print
Finally, we observe that batch normalization and deterministic warm-up (gradually turning on the KL-term) are crucial for training variational models with many stochastic layers.  ...  We propose a new inference model, the Ladder Variational Autoencoder, that recursively corrects the generative distribution by a data dependent approximate likelihood in a process resembling the recently  ...  We propose a new structured inference model using the same top-down dependency structure both in the inference and generative models.  ... 
arXiv:1602.02282v3 fatcat:53z5qnfysbfcpmo6stklz7vwfy

Deep Probabilistic Programming [article]

Dustin Tran, Matthew D. Hoffman, Rif A. Saurous, Eugene Brevdo, Kevin Murphy, David M. Blei
2017 arXiv   pre-print
For flexibility, Edward makes it easy to fit the same model using a variety of composable inference methods, ranging from point estimation to variational inference to MCMC.  ...  In addition, Edward can reuse the modeling representation as part of inference, facilitating the design of rich variational models and generative adversarial networks.  ...  As an example, Appendix B shows how to implement stochastic variational inference in Edward.  ... 
arXiv:1701.03757v2 fatcat:f3zxlird3bbpblw2fcrq3zuypm
« Previous Showing results 1 — 15 out of 170,184 results