Accelerated Stochastic Gradient Method for Composite Regularization

Wenliang Zhong, James Tin-Yau Kwok
2014 International Conference on Artificial Intelligence and Statistics  
Regularized risk minimization often involves nonsmooth optimization. This can be particularly challenging when the regularizer is a sum of simpler regularizers, as in the overlapping group lasso. Very recently, this is alleviated by using the proximal average, in which an implicitly nonsmooth function is employed to approximate the composite regularizer. In this paper, we propose a novel extension with accelerated gradient method for stochastic optimization. On both general convex and strongly
more » ... onvex problems, the resultant approximation errors reduce at a faster rate than methods based on stochastic smoothing and ADMM. This is also verified experimentally on a number of synthetic and real-world data sets.
dblp:conf/aistats/ZhongK14 fatcat:7a5i7mxsjjap3j74xnvc7cutua