10,378 Hits in 6.7 sec

Approximate Newton Methods [article]

Haishan Ye, Luo Luo, Zhihua Zhang
2020 arXiv   pre-print
We propose a unifying framework to analyze both local and global convergence properties of second order methods.  ...  Recently, subsampled Newton methods have emerged to attract much attention due to their efficiency at each iteration, rectified a weakness in the ordinary Newton method of suffering a high cost in each  ...  In Section 3 we present a unifying framework for local and global convergence analysis of second order methods.  ... 
arXiv:1702.08124v2 fatcat:2vp3ltpcjvc25ioljzalrxeszq

Approximate Newton Methods and Their Local Convergence

Haishan Ye, Luo Luo, Zhihua Zhang
2017 International Conference on Machine Learning  
We propose a unifying framework to analyze local convergence properties of second order methods. Based on this framework, our theoretical analysis matches the performance in real applications.  ...  Recently, stochastic second order methods have emerged to attract much attention for optimization due to their efficiency at each iteration, rectified a weakness in the ordinary Newton method of suffering  ...  Acknowledgements Ye has been supported by the National Natural Science Foundation of China (Grant No. 11426026, 61632017, 61173011)  ... 
dblp:conf/icml/YeLZ17 fatcat:46i2vsh275am5icryauicbwlni

Page 1350 of Mathematical Reviews Vol. 56, Issue 4 [page]

1978 Mathematical Reviews  
The author uses these concepts to present a unified theory of iterative methods that contain as particular cases the method of successive approxi- mations, the Newton-Kantorovi¢é method, the Newton-Altman  ...  The concept of a contractor is easily combined with the notion of majorant functions to unify and establish convergence theorems for iterative methods.  ... 

Page 6360 of Mathematical Reviews Vol. , Issue 99i [page]

1999 Mathematical Reviews  
The resulting Ritz pairs are approximations to the desired eigen- values of A. For the case in which A is normal, a convergence analysis for all three methods is given within a unified frame- work.  ...  Summary: “A unifying framework for methods employed in the approximation of the resolvent norm of nonnormal matrices is presented.  ... 

Page 4370 of Mathematical Reviews Vol. , Issue 99f [page]

1999 Mathematical Reviews  
Summary: “We present a unified framework for convergence anal- ysis of generalized subgradient-type algorithms in the presence of perturbations.  ...  Our analysis extends and unifies previously known results on convergence and stability properties of gradient and sub- gradient methods, including their incremental, parallel, and heavy ball modifications  ... 

Page 8461 of Mathematical Reviews Vol. , Issue 2002K [page]

2002 Mathematical Reviews  
Summary: “We study the choice of relaxation parameters w for convergence of the SOR Newton method and the SOR method for the system of equations F(x) = 0 in a unified framework, where F is strongly monotone  ...  The convergence analysis of the most frequently used simultaneous methods in multipoint mode is given.  ... 

The LQRE computational method in optimal control theory

Nikola B. Nedeljkovic
1986 Bulletin of the Australian Mathematical Society  
A proof of the reduction of the cost at each iterative step of the LQRE algorithms, a convergence analysis in the L space and a proof of the convergence in the space of relaxed controls are included.  ...  The method, named LQRE is modelled locally on the linear quadratic problem and treats a whole family of algorithms in a unified manner.  ... 
doi:10.1017/s0004972700010364 fatcat:5yywtp7hjzelhlb7crsau7l6lq

Revisiting Sub-sampled Newton Methods [article]

Haishan Ye, Luo Luo, Zhihua Zhang
2016 arXiv   pre-print
The proposed methods can be generalized to a unifying framework for the convergence proof of several existing sub-sampled Newton methods, revealing new convergence properties.  ...  Recently, sub-sampled Newton methods have emerged to attract much attention for optimization due to their efficiency at each iteration, rectified a weakness in the ordinary Newton method of suffering a  ...  The framework is a fundamental of convergence analysis of sub-sampled Newton methods.  ... 
arXiv:1608.02875v2 fatcat:6jiqdl5zv5chhb527p3qd4by74

Unified frameworks for high order Newton-Schulz and Richardson iterations: a computationally efficient toolkit for convergence rate improvement

Alexander Stotsky
2019 Journal of Applied Mathematics and Computing  
A trade-off between convergence rate, computational complexity and robustness necessitates development of unified framework for convergence rate improvement of high order Newton-Schulz algorithm.  ...  Two unified frameworks (1) for convergence rate improvement of high order Newton-Schulz matrix inversion algorithms and (2) for combination of Richardson and iterative matrix inversion algorithms with  ...  A unified framework for combined algorithms which takes the advantages of both approaches and includes high order Newton-Schulz algorithm and power series associated with (15) are designed in the next  ... 
doi:10.1007/s12190-018-01229-8 fatcat:g7p6p4iwzzfotlymbc46k4ylgi


S. Chen, Z. Xia, D. Yu, F. Meng
2013 International Journal of Pure and Applied Mathematics  
In this paper we apply the Monte Carlo method based on inexact Newton to solving a class of stochastic variational inequality problems (SVIPs).  ...  Locally convergence results of the proposed methods are proved under appropriate conditions.  ...  These two conditions provide a unified framework for inexact nonsmooth Newton schemes and we give the proof of locally convergence. The rest of the paper is organized as follows.  ... 
doi:10.12732/ijpam.v87i2.11 fatcat:qo3uvghs3ncunn5eooxbkfkyvm

A Unifying Perspective of Parametric Policy Search Methods for Markov Decision Processes

Thomas Furmston, David Barber
2012 Neural Information Processing Systems  
This analysis leads naturally to the consideration of this approximate Newton method as an alternative optimisation method for Markov Decision Processes.  ...  In this article we provide a unifying perspective of these two algorithms by showing that their searchdirections in the parameter space are closely related to the search-direction of an approximate Newton  ...  An Approximate Newton Method A natural follow on from the analysis in section(2) is the consideration of using M(w) = −H −1 2 (w) in (3), a method we call the full approximate Newton method from this point  ... 
dblp:conf/nips/FurmstonB12 fatcat:2zch52lcjbbzlcnjlzttgdyeqm

Fast and Exact Newton and Bidirectional Fitting of Active Appearance Models

Jean Kossaifi, Georgios Tzimiropoulos, Maja Pantic
2017 IEEE Transactions on Image Processing  
We derive both methods in a unified framework for two types of Active Appearance Models, holistic and part-based, and additionally show how to exploit the structure in the problem to derive fast yet exact  ...  We then formulate a second order method by deriving an efficient Newton method for AAMs fitting.  ...  UNIFIED OBJECTIVE FUNCTION To formulate our Newton and Bidirectional methods, we first introduce here a unified framework in which we derive all methods by formulating the problem of fitting an Active  ... 
doi:10.1109/tip.2016.2642828 pmid:28026767 fatcat:idqvbz2qkbafrithebmycflicm

Page 273 of Mathematical Reviews Vol. , Issue 81A [page]

1981 Mathematical Reviews  
Authors’ summary: “A unified framework is presented for analyz- ing the accuracy of finite difference, finite element, and spectral methods in approximating evolutionary problems.  ...  Slopes converge pointwise at the optimal rate O(h).” Girault, V.; Raviart, P.-A. An analysis of a mixed finite element method for the Navier-Stokes equations. Numer. Math, 33 (1979), no. 3, 235-271.  ... 

Page 4532 of Mathematical Reviews Vol. , Issue 92h [page]

1992 Mathematical Reviews  
These methods use the DAE system and some of its first derivatives as constraints to a least squares problem that corresponds to a Taylor series in y, or an approximative equality derived from a Padé approximation  ...  The derivation of the convergence properties is based on spectral analysis, rather than on the norm analysis.  ... 

Page 5869 of Mathematical Reviews Vol. , Issue 87j [page]

1987 Mathematical Reviews  
(BR-ECP) 87j:65059 A quasi-Newton method with a new updating for the LDU factorization of the approximate Jacobian. (Portuguese summary) Mat. Apl. Comput. 2 (1983), no. 2, 131-141.  ...  We prove a local linear convergence the- orem and that the convergence is superlinear for a version of the method with restarts. We show some numerical experiments.”  ... 
« Previous Showing results 1 — 15 out of 10,378 results