107 Hits in 5.1 sec

Stochastic Primal-Dual Method on Riemannian Manifolds with Bounded Sectional Curvature [article]

Masoud Badiei Khuzani, Na Li
2017 arXiv   pre-print
We study a stochastic primal-dual method for constrained optimization over Riemannian manifolds with bounded sectional curvature. We prove non-asymptotic convergence to the optimal objective value.  ...  More precisely, for the class of hyperbolic manifolds, we establish a convergence rate that is related to the sectional curvature lower bound.  ...  The research of MBK is supported by IBM PhD Fellowship.  ... 
arXiv:1703.08167v1 fatcat:tm6x5o7uhrhbljuuq5c2oiyyma

A Riemannian Primal-dual Algorithm Based on Proximal Operator and its Application in Metric Learning

Shijun Wang, Baocheng Zhu, Lintao Ma, Yuan Qi
2019 2019 International Joint Conference on Neural Networks (IJCNN)  
By utilizing the proposed primal-dual optimization technique, we propose a novel metric learning algorithm which learns an optimal feature transformation matrix in the Riemannian space of positive definite  ...  To solve the problem, we first convert it to a dual problem and then propose a general primal-dual algorithm to optimize the primal and dual variables iteratively.  ...  Previous Works Khuzani and Li studied stochastic primal-dual method on the Riemannian manifolds with bounded sectional curvature [24] .  ... 
doi:10.1109/ijcnn.2019.8852367 dblp:conf/ijcnn/WangZMQ19 fatcat:7ieyv35wmvh4levyeittrag6ne

Beyond Convexity – Contraction and Global Convergence of Gradient Descent [article]

Patrick M. Wensing, Jean-Jacques E. Slotine
2020 arXiv   pre-print
Riemannian manifold.  ...  Extensions to natural primal-dual optimization and game-theoretic contexts further illustrate the potential reach of these new perspectives.  ...  Recall that α-strong geodesic convexity of f (x, t) in the metric M(x) (for each t) is equivalent to the Riemannian Hessian of f , denoted H(x, t), satisfying:  ... 
arXiv:1806.06655v6 fatcat:mrpxa2d2jvcwhppxb7entbtxy4

An Elementary Introduction to Information Geometry

Frank Nielsen
2020 Entropy  
In this survey, we describe the fundamental differential-geometric structures of information manifolds, state the fundamental theorem of information geometry, and illustrate some use cases of these information  ...  manifolds in information sciences.  ...  We further explain the intrinsic curvature and torsion of manifolds induced by the connection in Section 2.3.4, and state the fundamental theorem of Riemannian geometry in Section 2.4: the existence of  ... 
doi:10.3390/e22101100 pmid:33286868 pmcid:PMC7650632 fatcat:ye2h3cjx35e6xcbrhiic2hmuea

First-Order Algorithms for Min-Max Optimization in Geodesic Metric Spaces [article]

Michael I. Jordan, Tianyi Lin, Emmanouil-Vasileios Vlatakis-Gkaragkounis
2022 arXiv   pre-print
Our results also extend to the stochastic or non-smooth case where RCEG and Riemanian gradient ascent descent (RGDA) achieve near-optimal convergence rates up to factors depending on curvature of the manifold  ...  From optimal transport to robust dimensionality reduction, a plethora of machine learning applications can be cast into the min-max optimization problems over Riemannian manifolds.  ...  Vlatakis-Gkaragkounis is grateful for financial support by the Google-Simons Fellowship, Pancretan Association of America and Simons Collaboration on Algorithms and Geometry.  ... 
arXiv:2206.02041v1 fatcat:o6mbfqgzbfcalhhpulervnxjnu

On the Curved Geometry of Accelerated Optimization [article]

Aaron Defazio
2019 arXiv   pre-print
By considering the optimization procedure as occurring on a Riemannian manifold with a natural structure, The AGM method can be seen as the proximal point method applied in this curved space.  ...  This viewpoint can also be extended to the continuous time case, where the accelerated gradient method arises from the natural block-implicit Euler discretization of an ODE on the manifold.  ...  We construct the AGM by applying a dual form of the proximal point method in a curved space. Each step follows a geodesic on a manifold in a sense we make precise in Section 4.  ... 
arXiv:1812.04634v2 fatcat:k3evepjyhrf2lphenw5oillw5m

An elementary introduction to information geometry [article]

Frank Nielsen
2020 arXiv   pre-print
In this survey, we describe the fundamental differential-geometric structures of information manifolds, state the fundamental theorem of information geometry, and illustrate some use cases of these information  ...  manifolds in information sciences.  ...  Curvature is a fundamental concept inherent to geometry [22]: There are several notions of curvatures: scalar curvature, sectional curvature, Gaussian curvature of surfaces to Riemannian-Christoffel 4-  ... 
arXiv:1808.08271v2 fatcat:q7cum6qk7ffbhpbmmyynvdeeya

Beyond convexity—Contraction and global convergence of gradient descent

Patrick M. Wensing, Jean-Jacques Slotine, Fei Chen
2020 PLoS ONE  
Riemannian manifold.  ...  Extensions to natural primal-dual optimization and game-theoretic contexts further illustrate the potential reach of these new perspectives.  ...  Primal dual dynamics in natural adaptive control This section illustrates the presence of natural primal-dual dynamics embedded in the application of natural adaptive control laws. s ¼ d dt þ λ � � nÀ  ... 
doi:10.1371/journal.pone.0236661 pmid:32750097 fatcat:dek5rhj7jnbq3o7ycfbg6rxhda

The Many Faces of Information Geometry

Frank Nielsen
2022 Notices of the American Mathematical Society  
𝑝 𝜃1 (𝑥) ent constant sectional curvatures 𝜅: for example, the curva- 𝒳 tures of the Fisher-Rao manifolds of univariate normal dis-  ...  Levi-Civita connection 𝑔 ∇ used by de- fault on a Riemannian manifold for obtaining (locally) length minimizing geodesics.  ... 
doi:10.1090/noti2403 fatcat:jvrvwvnyivd75cv7zow5s7fvfm

Momentum Improves Optimization on Riemannian Manifolds [article]

Foivos Alimisis, Antonio Orvieto, Gary Bécigneul, Aurelien Lucchi
2021 arXiv   pre-print
Our proofs of convergence rely on a novel estimate sequence that illustrates the dependency of the convergence rate on the curvature of the manifold.  ...  We develop a new Riemannian descent algorithm that relies on momentum to improve over existing first-order methods for geodesically convex optimization.  ...  Acknowledgements The authors would like to thank professors Nicolas Boumal and Suvrit Sra for helpful discussions regarding the content of this work.  ... 
arXiv:2002.04144v2 fatcat:rsj2vyvzdzgehfe3dpjtzotn4i

Averaging Stochastic Gradient Descent on Riemannian Manifolds [article]

Nilesh Tripuraneni, Nicolas Flammarion, Francis Bach, Michael I. Jordan
2018 arXiv   pre-print
We consider the minimization of a function defined on a Riemannian manifold M accessible only through unbiased estimates of its gradients.  ...  We develop a geometric framework to transform a sequence of slowly converging iterates generated from stochastic gradient descent (SGD) on M to an averaged iterate sequence with a robust and fast O(1/n  ...  Francis Bach acknowledges support from the European Research Council (grant SEQUOIA 724063), and Michael Jordan acknowledges support from the Mathematical Data Science program of the Office of Naval Research  ... 
arXiv:1802.09128v2 fatcat:6m7mcaminbgiff3vhqm72hoapy

Online and stochastic optimization beyond Lipschitz continuity: A Riemannian approach

Kimon Antonakopoulos, Elena Veronica Belmega, Panayotis Mertikopoulos
2020 International Conference on Learning Representations  
Drawing on tools and techniques from Riemannian geometry, we examine a Riemann-Lipschitz (RL) continuity condition which is tailored to the singularity landscape of the problem's loss functions.  ...  learning methods (such as online mirror descent).  ...  the Riemannian dual norm on the set ∞ t=1 β −2 t ¾[ζ 2 t | F t−1 ] < ∞.  ... 
dblp:conf/iclr/AntonakopoulosB20 fatcat:us43zkq3gnbrrgirw333ff2iay

Global Riemannian Acceleration in Hyperbolic and Spherical Spaces [article]

David Martínez-Rubio
2022 arXiv   pre-print
Additionally, for any Riemannian manifold of bounded sectional curvature, we provide reductions from optimization methods for smooth and g-convex functions to methods for smooth and strongly g-convex functions  ...  We further research on the accelerated optimization phenomenon on Riemannian manifolds by introducing accelerated global first-order methods for the optimization of L-smooth and geodesically convex (g-convex  ...  Acknowledgments We thank Mario Lezcano-Casado for helpful discussions on this work. We thank Varun Kanade and Patrick Rebeschini for proofreading of this work.  ... 
arXiv:2012.03618v4 fatcat:dymhmosw6nh4finmhfnzzraxuu

KKT Conditions, First-Order and Second-Order Optimization, and Distributed Optimization: Tutorial and Survey [article]

Benyamin Ghojogh, Ali Ghodsi, Fakhri Karray, Mark Crowley
2021 arXiv   pre-print
After a brief review of history of optimization, we start with some preliminaries on properties of sets, norms, functions, and concepts of optimization.  ...  We introduce Lagrangian function, dual variables, KKT conditions (including primal feasibility, dual feasibility, weak and strong duality, complementary slackness, and stationarity condition), and solving  ...  Stephen Boyd for his great courses Convex Optimization 1 and 2 of Stanford University available on YouTube (The course Convex Optimization 1 mostly focuses on second-order and interior-point methods and  ... 
arXiv:2110.01858v1 fatcat:4zz2gdfk75e6finemlbnda43ui

Hyperbolic Deep Neural Networks: A Survey [article]

Wei Peng, Tuomas Varanka, Abdelrahman Mostafa, Henglin Shi, Guoying Zhao
2021 arXiv   pre-print
It also presents current applicationsaround various machine learning tasks on several publicly available datasets, together with insightful observations and identifying openquestions and promising future  ...  generalization of the leading deep approaches to the Hyperbolic space.  ...  This work is supported by the Academy of Finland for ICT 2023 project (grant 328115) and project MiGA (grant 316765) and Infotech Oulu.  ... 
arXiv:2101.04562v3 fatcat:yqj4zohrqjbplpsdy5f5uglnbu
« Previous Showing results 1 — 15 out of 107 results