157 Hits in 4.8 sec

Asymptotic Guarantees for Learning Generative Models with the Sliced-Wasserstein Distance [article]

Kimia Nadjahi, Alain Durmus, Umut Şimşekli, Roland Badeau
2020 arXiv   pre-print
Minimum expected distance estimation (MEDE) algorithms have been widely used for probabilistic models with intractable likelihood functions and they have become increasingly popular due to their use in  ...  Emerging from computational optimal transport, the Sliced-Wasserstein (SW) distance has become a popular choice in MEDE thanks to its simplicity and computational benefits.  ...  This work is partly supported by the French National Research Agency (ANR) as a part of the FBIMATRIX project (ANR-16-CE23-0014) and by the industrial chair Machine Learning for Big Data from Télécom ParisTech  ... 
arXiv:1906.04516v2 fatcat:f6ox4wmilncfveodwc5f72bdei

Approximate Bayesian Computation with the Sliced-Wasserstein Distance [article]

Kimia Nadjahi, Valentin De Bortoli, Alain Durmus, Roland Badeau, Umut Şimşekli
2020 arXiv   pre-print
Approximate Bayesian Computation (ABC) is a popular method for approximate inference in generative models with intractable but easy-to-sample likelihood.  ...  We propose a new ABC technique, called Sliced-Wasserstein ABC and based on the Sliced-Wasserstein distance, which has better computational and statistical properties.  ...  However, each iteration requires n 2 operations, and there is no guarantee of convergence to Wp. SLICED-WASSERSTEIN ABC Sliced-Wasserstein distance.  ... 
arXiv:1910.12815v2 fatcat:p7u6viu3lzgyjdho2rtk5lwtmm

Hilbert Sinkhorn Divergence for Optimal Transport

Qian Li, Zhichao Wang, Gang Li, Jun Pang, Guandong Xu
2021 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)  
It is therefore of theoretical demand to empower the Sinkhorn divergence with the capability of capturing nonlinear structures.  ...  We also prove several attractive statistical properties of the proposed HSD, i.e., strong consistency, asymptotic behavior and sample complexity.  ...  In other words, the sliced Wasserstein distance is calculated via linear slicing of the probability distribution.  ... 
doi:10.1109/cvpr46437.2021.00383 fatcat:445micqmkrcjzpsvgaiyloswni

Minimax Confidence Intervals for the Sliced Wasserstein Distance [article]

Tudor Manole, Sivaraman Balakrishnan, Larry Wasserman
2020 arXiv   pre-print
Motivated by the growing popularity of variants of the Wasserstein distance in statistics and machine learning, we study statistical inference for the Sliced Wasserstein distance--an easily computable  ...  Specifically, we construct confidence intervals for the Sliced Wasserstein distance which have finite-sample validity under no assumptions or under mild moment assumptions.  ...  Acknowledgments Tudor Manole would like to thank Niccolò Dalmasso for conversations which inspired the likelihood-free inference application in Section 7.  ... 
arXiv:1909.07862v2 fatcat:dhe2ml366nerfodhwr32nv6kvi

Tessellated Wasserstein Auto-Encoders [article]

Kuo Gai, Shihua Zhang
2021 arXiv   pre-print
Non-adversarial generative models such as variational auto-encoder (VAE), Wasserstein auto-encoders with maximum mean discrepancy (WAE-MMD), sliced-Wasserstein auto-encoder (SWAE) are relatively easy to  ...  train and have less mode collapse compared to Wasserstein auto-encoder with generative adversarial network (WAE-GAN).  ...  The Wasserstein distance is a true distance and has a finer topology to guarantee convergence when minimize the distance.  ... 
arXiv:2005.09923v2 fatcat:cg3qnrznznfqbpbzopk4tzdcc4

Learning with minibatch Wasserstein : asymptotic and gradient properties [article]

Kilian Fatras, Younes Zine, Rémi Flamary, Rémi Gribonval, Nicolas Courty
2021 arXiv   pre-print
, but also with defects such as loss of distance property.  ...  Optimal transport distances are powerful tools to compare probability distributions and have found many applications in machine learning.  ...  Acknowledgements Authors would like to thank Thibault Séjourné and Jean Feydy for fruitful discussions.  ... 
arXiv:1910.04091v4 fatcat:wqdqgos4kbh7xa32lf5sefws3u

Exploiting Chain Rule and Bayes' Theorem to Compare Probability Distributions [article]

Huangjie Zheng, Mingyuan Zhou
2021 arXiv   pre-print
On a wide variety of benchmark datasets for generative modeling, substituting the default statistical distance of an existing generative adversarial network with CT is shown to consistently improve the  ...  When applied to train a generative model, CT is shown to strike a good balance between mode-covering and mode-seeking behaviors and strongly resist mode collapse.  ...  CT with feature space cooperatively trained with discriminator loss. (d-f) Sliced Wasserstein distance and CT in the sliced space.  ... 
arXiv:2012.14100v5 fatcat:u3qhz4lok5dpnbhudcnjtgjcua

Statistical Analysis of Wasserstein Distributionally Robust Estimators [article]

Jose Blanchet and Karthyek Murthy and Viet Anh Nguyen
2021 arXiv   pre-print
Equipped with this prescription, we present a central limit theorem for the DRO estimator and provide a recipe for constructing compatible confidence regions that are useful for uncertainty quantification  ...  for optimally selecting the size of the adversary's budget.  ...  [50] , tree-sliced Wasserstein distance [55] , etc.  ... 
arXiv:2108.02120v1 fatcat:dadtqnmvl5bk7phxmqba2aesbi

Fast Approximation of the Sliced-Wasserstein Distance Using Concentration of Random Projections [article]

Kimia Nadjahi, Alain Durmus, Pierre E. Jacob, Roland Badeau, Umut Şimşekli
2022 arXiv   pre-print
The Sliced-Wasserstein distance (SW) is being increasingly used in machine learning applications as an alternative to the Wasserstein distance and offers significant computational and statistical benefits  ...  We validate our theoretical findings on synthetic datasets, and illustrate the proposed approximation on a generative modeling problem.  ...  Acknowledgments This work is partly supported by the industrial chair "Data Science & Artificial Intelligence for Digitalized Industry & Services" from Télécom Paris. U.Ş.'  ... 
arXiv:2106.15427v2 fatcat:fdsbbghdbffh3dpfyb2s3lyvmu

OUP accepted manuscript

2019 Information and Inference A Journal of the IMA  
Statistical inference can be performed by minimizing, over the parameter space, the Wasserstein distance between model distributions and the empirical distribution of the data.  ...  We study asymptotic properties of such minimum Wasserstein distance estimators, complementing results derived by Bassetti, Bodini and Regazzini in 2006.  ...  Acknowledgements The bootstrap experiments were in part performed on the Odyssey cluster supported by the FAS Division of Science, Research Computing Group at Harvard University. Pierre E.  ... 
doi:10.1093/imaiai/iaz003 fatcat:s6umcdcc6rfzbjrrd2borgpdpm

Controlling Wasserstein distances by Kernel norms with application to Compressive Statistical Learning [article]

Titouan Vayer, Rémi Gribonval
2021 arXiv   pre-print
Based on the relations between the MMD and the Wasserstein distance, we provide guarantees for compressive statistical learning by introducing and studying the concept of Wasserstein learnability of the  ...  Our work is motivated by the compressive statistical learning (CSL) theory, a general framework for resource-efficient large scale learning in which the training data is summarized in a single vector (  ...  Asymptotic guarantees for generative modeling based on the smooth wasserstein distance. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Bal- can, and H.  ... 
arXiv:2112.00423v2 fatcat:hnlpmfoxjjhlln22623nn2nluq

Improving Relational Regularized Autoencoders with Spherical Sliced Fused Gromov Wasserstein [article]

Khai Nguyen and Son Nguyen and Nhat Ho and Tung Pham and Hung Bui
2020 arXiv   pre-print
Relational regularized autoencoder (RAE) is a framework to learn the distribution of data by minimizing a reconstruction loss together with a relational regularization on the latent space.  ...  That approach has a weakness since it treats every slicing direction similarly, meanwhile several directions are not useful for the discriminative task.  ...  learning models [29] .  ... 
arXiv:2010.01787v1 fatcat:t3clf5oqhfe45p57vspskmbuli

Hypothesis Test and Confidence Analysis with Wasserstein Distance on General Dimension [article]

Masaaki Imaizumi, Hirofumi Ota, Takuo Hamaguchi
2022 arXiv   pre-print
We develop a general framework for statistical inference with the 1-Wasserstein distance.  ...  To address this problem, in this study, we develop a novel non-asymptotic Gaussian approximation for the empirical 1-Wasserstein distance.  ...  Owing to this advantage, the Wasserstein distance has been utilized extensively in machine learning and related fields, such as in generative models (Arjovsky et al., 2017) , supervised learning (Frogner  ... 
arXiv:1910.07773v3 fatcat:bsxkjpfej5cvlh6dxwdht7jxpy

On Projection Robust Optimal Transport: Sample Complexity and Model Misspecification [article]

Tianyi Lin, Zeyu Zheng, Elynn Y. Chen, Marco Cuturi, Michael I. Jordan
2021 arXiv   pre-print
Optimal transport (OT) distances are increasingly used as loss functions for statistical inference, notably in the learning of generative models or supervised learning.  ...  We provide an asymptotic guarantee of two types of minimum PRW estimators and formulate a central limit theorem for max-sliced Wasserstein estimator under model misspecification.  ...  A generic Riemannian supergradient ascent algorithm for solving this problem is given by U t+1 ← Retr Ut (γ t+1 ξ t+1 ) for any ξ t+1 ∈ subdiff F (U t ),  ... 
arXiv:2006.12301v5 fatcat:kvuhrbldizdltfhd4kotkmkbqi

Complementary Learning for Overcoming Catastrophic Forgetting Using Experience Replay [article]

Mohammad Rostami, Soheil Kolouri, Praveen K. Pilly
2019 arXiv   pre-print
Inspired from complementary learning systems theory, we address this challenge by learning a generative model that couples the current task to the past learned tasks through a discriminative embedding  ...  We learn an abstract level generative distribution in the embedding that allows the generation of data points to represent the experience.  ...  Acknowledgment We thank James McClelland, Amarjot Singh, Charles Martin, Nicholas Ketz, and Jeffrey Krichmar for helpful feedback in the development and analysis of this work and conceptual discussions  ... 
arXiv:1903.04566v2 fatcat:ctffog6qgvgbvd5kipd3ddku7a
« Previous Showing results 1 — 15 out of 157 results