42,302 Hits in 3.1 sec

Communication Efficient Distributed Optimization using an Approximate Newton-type Method [article]

Ohad Shamir, Nathan Srebro, Tong Zhang
2014 arXiv   pre-print
We present a novel Newton-type method for distributed optimization, which is particularly well suited for stochastic optimization and learning problems.  ...  We provide theoretical and empirical evidence of the advantages of our method compared to other approaches, such as one-shot parameter averaging and ADMM.  ...  Distributed Approximate Newton-type Method We now describe a new iterative method for distributed optimization.  ... 
arXiv:1312.7853v4 fatcat:jwj7s4an25crhgtmbxh7hq7zty

Distributed Sensor Selection using a Truncated Newton Method [article]

Danny Bickson, Danny Dolev
2010 arXiv   pre-print
We propose a new distributed algorithm for computing a truncated Newton method, where the main diagonal of the Hessian is computed using belief propagation.  ...  The first algorithm is a distributed version of the interior point method by Joshi and Boyd, and the second algorithm is an order of magnitude faster approximation.  ...  The approximated function (4) is concave and smooth and thus it is possible to use the Newton method [1, §10.2.2] efficiently. Table I -A outlines the constrained Newton method.  ... 
arXiv:0907.0931v2 fatcat:xtyu7fyoavhfxjguipu2bwbfae

Newton-ADMM: A Distributed GPU-Accelerated Optimizer for Multiclass Classification Problems [article]

Chih-Hao Fang, Sudhir B Kylasa, Fred Roosta, Michael W. Mahoney, Ananth Grama
2020 arXiv   pre-print
By leveraging the communication efficiency of ADMM, GPU-accelerated inexact-Newton solver, and an effective spectral penalty parameter selection strategy, we show that our proposed method (i) yields better  ...  In contrast, Newton-type methods, while having higher per-iteration costs, typically require a significantly smaller number of iterations, which directly translates to reduced communication costs.  ...  Newton-type method as a highly efficient subproblem solver for ADMM.  ... 
arXiv:1807.07132v3 fatcat:s2pmy52krvd7fiz2k7mvbpgg5i

Distributed Newton-Based Voltage Control Method for High-Penetration PV Generation Cluster in Active Distribution Networks

Zhongguan Wang, Leijiao Ge, Jinjin Ding, Weide Gu
2020 IET Renewable Power Generation  
In this paper, a distributed Newton-based voltage control method for large-scale PV generation cluster in distribution networks is presented to realize distributed coordination of PV inverters, which is  ...  based on matrix splitting and approximate Newton iteration, and can fast respond to reactive power mismatch and realize voltage profiles optimization.  ...  Distributed Newton method Approximate Newton iteration According to (19) , we can obtain the gradient of the objective: g = X(Xq g − Ṽ) = X(V − μ) ≃ X(V − μ) (21) where X denotes a matrix which only  ... 
doi:10.1049/iet-rpg.2020.0163 fatcat:467xwycrire6hl4ya6kyqo7cwu

Distributed Adaptive Newton Methods with Global Superlinear Convergence [article]

Jiaqi Zhang, Keyou You, Tamer Başar
2022 arXiv   pre-print
In sharp contrast to the existing literature where the fastest distributed algorithms converge either with a global linear or a local superlinear rate, we propose a distributed adaptive Newton (DAN) algorithm  ...  This paper considers the distributed optimization problem where each node of a peer-to-peer network minimizes a finite sum of objective functions by communicating with its neighboring nodes.  ...  Communication-efficient distributed optimization Zhang, J. & You, K. (2019a), ‘AsySPA: An exact asyn- using an approximate Newton-type method, in chronous algorithm for convex  ... 
arXiv:2002.07378v3 fatcat:fopdg6h3ibhlvn7q4qr2wm6zpi

An inexact-Newton method for short-range microwave imaging within the second-order Born approximation

C. Estatico, M. Pastorino, A. Randazzo
2005 IEEE Transactions on Geoscience and Remote Sensing  
The data inversion is performed by applying an inexact-Newton method (which exhibits very good regularization properties) within the second-order Born approximation [2] .  ...  In this work, a novel electromagnetic inverse scattering method is proposed for the reconstruction of shallow buried objects.  ...  The data inversion is performed by applying an inexact-Newton method (which exhibits very good regularization properties) within the second-order Born approximation [2] .  ... 
doi:10.1109/tgrs.2005.856631 fatcat:ikq3vynuxnbmvfzwnfqjajcsam

Distributed Newton Methods for Regularized Logistic Regression [chapter]

Yong Zhuang, Wei-Sheng Chin, Yu-Chin Juan, Chih-Jen Lin
2015 Lecture Notes in Computer Science  
In this work, we propose a distributed Newton method for training logistic regression. Many interesting techniques are discussed for reducing the communication cost and speeding up the computation.  ...  Regularized logistic regression is a very useful classification method, but for large-scale data, its distributed training has not been investigated much.  ...  Distributed Newton Methods In this section, we describe our proposed implementation of a distributed Newton method. Newton Methods We denote the objective function of (1) as f (w).  ... 
doi:10.1007/978-3-319-18032-8_54 fatcat:oz4ab7phnrg3hgf6vjgzikbxci

Newton-like Method with Diagonal Correction for Distributed Optimization

Dragana Bajović, Dušan Jakovetić, Nataša Krejić, Nataša Krklec Jerinkić
2017 SIAM Journal on Optimization  
We overcome this challenge and propose a class of distributed Newton-like methods, which we refer to as Distributed Quasi Newton (DQN).  ...  Specific choices of the tuning variables give rise to different variants of the proposed general DQN method -dubbed DQN-0, DQN-1 and DQN-2 -which mutually trade-off communication and computational costs  ...  Network Newton method [19] aims at solving (1) and presents a family of distributed (approximate) Newton methods.  ... 
doi:10.1137/15m1038049 fatcat:pjsw5ol6izfeti7ouvqm5gjvnm

On using newton's method for computing the noncentrality parameter of the noncentral f distribution

Cherng G. Ding
1997 Communications in statistics. Simulation and computation  
This article deals with how to enhance computational efficiency of Newton's method for computing the noncentrality parameter of the noncentral F distribution.  ...  An efficient algorithm featuring this finding is provided in a step-by-step form. 259  ...  The efficiency of Newton's method can be greatly enhanced by the treatments mentioned above.  ... 
doi:10.1080/03610919708813377 fatcat:5yvrzfs3hndtpnb2oefgfsjmea

GIANT: Globally Improved Approximate Newton Method for Distributed Optimization [article]

Shusen Wang, Farbod Roosta-Khorasani, Peng Xu, Michael W. Mahoney
2018 arXiv   pre-print
For distributed computing environment, we consider the empirical risk minimization problem and propose a distributed and communication-efficient Newton-type optimization method.  ...  Theoretically, we show that GIANT enjoys an improved convergence rate as compared with first-order methods and existing distributed Newton-type methods.  ...  Compared Methods We compare GIANT with three methods: Accelerated Gradient Descent (AGD) [33] , Limited memory BFGS (L-BFGS) [18] , and Distributed Approximate NEwton (DANE) [46] .  ... 
arXiv:1709.03528v5 fatcat:bymn2vkpvrdulpntqa245dh6lm

PrivLogit: Efficient Privacy-preserving Logistic Regression by Tailoring Numerical Optimizers [article]

Wei Xie, Yang Wang, Steven M. Boker, Donald E. Brown
2016 arXiv   pre-print
., Newton method) and failing to tailor for secure computing. This work presents a contrasting perspective: customizing numerical optimization specifically for secure settings.  ...  Leveraging this new method, we propose two new secure protocols for conducting logistic regression in a privacy-preserving and distributed manner.  ...  The standard non-secure distributed Newton method serves as the ground truth.  ... 
arXiv:1611.01170v1 fatcat:znqerhrw6rhojlgnmfkfienphy

OverSketched Newton: Fast Convex Optimization for Serverless Systems [article]

Vipul Gupta, Swanand Kadhe, Thomas Courtade, Michael W. Mahoney, Kannan Ramchandran
2020 arXiv   pre-print
OverSketched Newton leverages matrix sketching ideas from Randomized Numerical Linear Algebra to compute the Hessian approximately.  ...  These sketching methods lead to inbuilt resiliency against stragglers that are a characteristic of serverless architectures.  ...  We note that it is well known in HPC that blocked partitioning of input matrices can lead to communication-efficient methods for distributed multiplication [32, 49, 50] .  ... 
arXiv:1903.08857v3 fatcat:luuf7mcdm5dk3l4bhpmf4khmq4

Efficient Distributed Hessian Free Algorithm for Large-scale Empirical Risk Minimization via Accumulating Sample Strategy [article]

Majid Jahani, Xi He, Chenxin Ma, Aryan Mokhtari, Dheevatsa Mudigere, Alejandro Ribeiro, Martin Takáč
2020 arXiv   pre-print
In this paper, we propose a Distributed Accumulated Newton Conjugate gradiEnt (DANCE) method in which sample size is gradually increasing to quickly obtain a solution whose empirical loss is under satisfactory  ...  Various iteration complexity results regarding descent direction computation, communication efficiency and stopping criteria are analyzed under convex setting.  ...  Distributed Accumulated Newton Conjugate Gradient Method The goal in inexact damped Newton method, as discussed in [Zhang and Lin, 2015] , is to find the next iterate based on an approximated Newton-type  ... 
arXiv:1810.11507v2 fatcat:u2lrfsydj5glnjelwktxmw2jjy

DAve-QN: A Distributed Averaged Quasi-Newton Method with Local Superlinear Convergence Rate [article]

Saeed Soori, Konstantin Mischenko, Aryan Mokhtari, Maryam Mehri Dehnavi, Mert Gurbuzbalaban
2019 arXiv   pre-print
We develop a distributed asynchronous quasi-Newton algorithm that can achieve superlinear convergence.  ...  Our algorithm is communication-efficient in the sense that at every iteration the master node and workers communicate vectors of size O(p), where p is the dimension of the decision variable.  ...  Unlike distributed first-order methods that follow the gradient direction to update the iterates, we proposed a distributed averaged quasi-Newton (DAve-QN) algorithm that uses a quasi-Newton approximate  ... 
arXiv:1906.00506v3 fatcat:wjbgz3jb7zao7bjh6hupkboe4m

Page 3588 of Mathematical Reviews Vol. , Issue 2000e [page]

2000 Mathematical Reviews  
In the paper, a Davidson and an Arnoldi approach are designed to approximate the dominant eigenvalues in an efficient way.  ...  The second algorithm is a Newton iteration for simultaneously improving the accuracy of all factors in an approximate factorization of a polynomial, respectively, all partial fractions of an approximate  ... 
« Previous Showing results 1 — 15 out of 42,302 results