55 Hits in 2.6 sec

Coresets for Scalable Bayesian Logistic Regression [article]

Jonathan H. Huggins, Trevor Campbell, Tamara Broderick
2017 arXiv   pre-print
In this paper, we develop an efficient coreset construction algorithm for Bayesian logistic regression models.  ...  We provide theoretical guarantees on the size and approximation quality of the coreset -- both for fixed, known datasets, and in expectation for a wide class of data generative models.  ...  Although our coreset algorithm is specifically for logistic regression, our approach is broadly applicable to other Bayesian generative models.  ... 
arXiv:1605.06423v3 fatcat:7h27aedambbujlurg4sfbz42im

Automated Scalable Bayesian Inference via Hilbert Coresets [article]

Trevor Campbell, Tamara Broderick
2019 arXiv   pre-print
Building on the Bayesian coresets framework, this work instead takes advantage of data redundancy to shrink the dataset itself as a preprocessing step, providing fully-automated, scalable Bayesian inference  ...  To address these shortcomings we develop Hilbert coresets, i.e., Bayesian coresets constructed under a norm induced by an inner-product on the log-likelihood function space.  ...  We also thank Sushrutha Reddy for finding and correcting a bug in the proof of Theorem 3.2.  ... 
arXiv:1710.05053v2 fatcat:n2fyqlemdjeu3jqtcckee3tfgm

β-Cores: Robust Large-Scale Bayesian Data Summarization in the Presence of Outliers [article]

Dionysis Manousakas, Cecilia Mascolo
2020 arXiv   pre-print
Moreover, relying on the recent formulations of Riemannian coresets for scalable Bayesian inference, we propose a sparse variational approximation of the robustified posterior and an efficient stochastic  ...  We illustrate the applicability of our approach in diverse simulated and real datasets, and various statistical models, including Gaussian mean inference, logistic and neural linear regression, demonstrating  ...  Figure 2 : 2 Predictive accuracy vs coreset size for logistic regression experiments over 10 trials on 3 large-scale datasets.  ... 
arXiv:2008.13600v2 fatcat:65hmo56nn5fpvkc3ggsv5alihi

Bayesian Coresets: Revisiting the Nonconvex Optimization Perspective [article]

Jacky Y. Zhang, Rajiv Khanna, Anastasios Kyrillidis, Oluwasanmi Koyejo
2021 arXiv   pre-print
Bayesian coresets have emerged as a promising approach for implementing scalable Bayesian inference.  ...  Leveraging recent advances in accelerated optimization methods, we propose and analyze a novel algorithm for coreset selection.  ...  Introduction Bayesian coresets have emerged as a promising approach for scalable Bayesian inference [22, 12, 13, 11] .  ... 
arXiv:2007.00715v2 fatcat:awoiiwz5jnbrbenlva4cdc3dmq

Sparse Variational Inference: Bayesian Coresets from Scratch [article]

Trevor Campbell, Boyan Beronov
2019 arXiv   pre-print
Recent work on Bayesian coresets takes the approach of compressing the dataset before running a standard inference algorithm, providing both scalability and guarantees on posterior approximation error.  ...  , the proposed algorithm is able to continually improve the coreset, providing state-of-the-art Bayesian dataset summarization with orders-of-magnitude reduction in KL divergence to the exact posterior  ...  Bayesian logistic and Poisson regression Finally, we compared the methods on logistic and Poisson regression applied to six datasets (details may be found in Appendix C) with N = 500 and dimension ranging  ... 
arXiv:1906.03329v2 fatcat:t3gfmln6uncvroymmgbuz4tl2q

Bayesian Coreset Construction via Greedy Iterative Geodesic Ascent [article]

Trevor Campbell, Tamara Broderick
2018 arXiv   pre-print
But modern algorithms for approximate Bayesian posterior inference often sacrifice accurate posterior uncertainty estimation in the pursuit of scalability.  ...  To address this shortcoming, we develop greedy iterative geodesic ascent (GIGA), a novel algorithm for Bayesian coreset construction that scales the coreset log-likelihood optimally.  ...  Bayesian Posterior Approximation In this experiment, we used GIGA to generate Bayesian coresets for logistic and Poisson regression.  ... 
arXiv:1802.01737v2 fatcat:myl4ewuszrfdrgr76gxf4zoa7a

A Novel Sequential Coreset Method for Gradient Descent Algorithms [article]

Jiawei Huang, Ruomin Huang, Wenjie Liu, Nikolaos M. Freris, Hu Ding
2021 arXiv   pre-print
However, most of existing coreset methods are problem-dependent and cannot be used as a general tool for a broader range of applications.  ...  Moreover, our method is particularly suitable for sparse optimization whence the coreset size can be further reduced to be only poly-logarithmically dependent on the dimension.  ...  Coresets for scalable bayesian logistic regression. In Advances in Neural Information Processing Systems, pages 4080–4088, 2016. [27] Praneeth Kacham and David P. Woodruff.  ... 
arXiv:2112.02504v1 fatcat:ca6ik4vfgfgqrmoqottujcbhwa

Text/Conference Paper

Alexander Munteanu, Chris Schwiegelshohn, Christian Sohler, David P. Woodruff
2019 Jahrestagung der Gesellschaft für Informatik  
First, we show the negative result that no strongly sublinear sized coresets exist for logistic regression.  ...  Coresets are one of the central methods to facilitate the analysis of large data.We continue a recent line of research applying the theory of coresets to logistic regression.  ...  Acknowledgments We thank the anonymous reviewers for their valuable comments. We also thank our assistant Moritz Paweletz.  ... 
doi:10.18420/inf2019_37 dblp:conf/gi/MunteanuSSW19 fatcat:7smn24bxybblrlo2qzjc2ykvq4

Data Summarization via Bilevel Optimization [article]

Zalán Borsos, Mojmír Mutný, Marco Tagliasacchi, Andreas Krause
2021 arXiv   pre-print
However, existing coreset constructions are highly model-specific and are limited to simple models such as linear regression, logistic regression, and k-means.  ...  Coresets are weighted subsets of the data that provide approximation guarantees for the optimization objective.  ...  We standardize the features and solve Figure 6 : Coresets for binary logistic regression.  ... 
arXiv:2109.12534v1 fatcat:f5yewtrb3nehfcs2s5i6jxbfgy

Generic Coreset for Scalable Learning of Monotonic Kernels: Logistic Regression, Sigmoid and more [article]

Elad Tolochinsky, Ibrahim Jubran, Dan Feldman
2021 arXiv   pre-print
(ii) A proof that, under a natural assumption that holds e.g. for logistic regression and the sigmoid activation functions, a small coreset exists for any input P.  ...  In this work we provide: (i) A lower bound which proves that there are sets with no coresets smaller than n=|P| for general monotonic loss functions.  ...  The following theorems construct a coreset for sums of sigmoid functions and for the logistic regression log-likelihood, for normalized input sets.  ... 
arXiv:1802.07382v3 fatcat:7lapvbmpubal5gj3e2j662637e

On Coresets for Logistic Regression [article]

Alexander Munteanu and Chris Schwiegelshohn and Christian Sohler and David P. Woodruff
2018 arXiv   pre-print
First, we show a negative result, namely, that no strongly sublinear sized coresets exist for logistic regression.  ...  The experiments are conducted on real world benchmark data for logistic regression.  ...  Acknowledgments We thank the anonymous reviewers for their valuable comments. We also thank our student assistant Moritz Paweletz for implementing and conducting the experiments.  ... 
arXiv:1805.08571v2 fatcat:z4fw6euzrbahhe3ajg7u22cuaq

Bayesian Batch Active Learning as Sparse Subset Approximation [article]

Robert Pinsler, Jonathan Gordon, Eric Nalisnick, José Miguel Hernández-Lobato
2021 arXiv   pre-print
In this paper, we introduce a novel Bayesian batch active learning approach that mitigates these issues.  ...  We demonstrate the benefits of our approach on several large-scale regression and classification tasks.  ...  We thank Adrià Garriga-Alonso, James Requeima, Marton Havasi, Carl Edward Rasmussen and Trevor Campbell for helpful feedback and discussions.  ... 
arXiv:1908.02144v4 fatcat:zovwsi2oenftzfm3jqwl3habdi

Surrogate Likelihoods for Variational Annealed Importance Sampling [article]

Martin Jankowiak, Du Phan
2021 arXiv   pre-print
Variational inference is a powerful paradigm for approximate Bayesian inference with a number of appealing properties, including support for model learning and data subsampling.  ...  For these reasons researchers have sought to combine the strengths of both classes of algorithms, with recent approaches coming closer to realizing this vision in practice.  ...  Hand- scalable bayesian logistic regression. Advances in Neural book of markov chain monte carlo, 2(11):2, 2011.  ... 
arXiv:2112.12194v1 fatcat:2qhyj7y73bdtvfxh2frq7s6jme

Interpreting Black Box Predictions using Fisher Kernels [article]

Rajiv Khanna, Been Kim, Joydeep Ghosh, Oluwasanmi Koyejo
2018 arXiv   pre-print
To answer this question, we make use of Fisher kernels as the defining feature embedding of each data point, combined with Sequential Bayesian Quadrature (SBQ) for efficient selection of examples.  ...  Our goal is to ask 'which training examples are most responsible for a given set of predictions'?  ...  Figure 4 : Performance for logistic regression over two datasets (left is ChemReact while right is CovType) of our method (Fisher) vs coreset selection [11] and random data selection.  ... 
arXiv:1810.10118v1 fatcat:ii35eupdejfuzn6kbftueffdni

p-Generalized Probit Regression and Scalable Maximum Likelihood Estimation via Sketching and Coresets [article]

Alexander Munteanu, Simon Omlor, Christian Peters
2022 arXiv   pre-print
We study the p-generalized probit regression model, which is a generalized linear model for binary responses.  ...  subsampling to obtain a small data summary called coreset.  ...  Katja Ickstadt for pointing us to the probit model and for valuable discussions on that topic.  ... 
arXiv:2203.13568v1 fatcat:fdbiroscdrderfn5ftphdxevoa
« Previous Showing results 1 — 15 out of 55 results