3 Hits in 2.2 sec

On Margins and Derandomisation in PAC-Bayes [article]

Felix Biggs, Benjamin Guedj
2022 arXiv   pre-print
We give a general recipe for derandomising PAC-Bayesian bounds using margins, with the critical ingredient being that our randomised predictions concentrate around some value.  ...  The tools we develop straightforwardly lead to margin bounds for various classifiers, including linear prediction -- a class that includes boosting and the support vector machine -- single-hidden-layer  ...  this derive (partially) derandomised PAC-Bayesian margin bounds in Sec. 2.2.  ... 
arXiv:2107.03955v3 fatcat:zdhfdvrtcjh25jdn7fqql6ls2i

Kernel Interpolation as a Bayes Point Machine [article]

Jeremy Bernstein and Alex Farhang and Yisong Yue
2022 arXiv   pre-print
Since large margin, infinite width neural networks are kernel interpolators, the paper's findings may help to explain generalisation in neural networks more broadly.  ...  Supporting this idea, the paper finds evidence that large margin, finite width neural networks behave like Bayes point machines too.  ...  PAC-Bayes theory.  ... 
arXiv:2110.04274v2 fatcat:ub3rwfg7hndutpzzwflbebfwwi

On PAC-Bayesian reconstruction guarantees for VAEs

Badr-Eddine Chérief-Abdellatif, Yuyang Shi, Arnaud Doucet, Benjamin Guedj
We contribute to this recent line of work by analysing the VAE's reconstruction ability for unseen test data, leveraging arguments from the PAC-Bayes theory.  ...  Despite its wide use and empirical successes, the theoretical understanding and study of the behaviour and performance of the variational autoencoder (VAE) have only emerged in the past few years.  ...  Acknowledgements Badr-Eddine Chérief-Abdellatif, Benjamin Guedj and Arnaud Doucet acknowledge support of the UK Defence Science and Technology Laboratory (DSTL) and EPSRC under grants EP/R018693/1 and  ... 
doi:10.48550/arxiv.2202.11455 fatcat:w2ks4wybirfhpgonnfi7xllxdi