Convex vs nonconvex approaches for sparse estimation: Lasso, Multiple Kernel Learning and Hyperparameter Lasso

Aleksander Aravkin, James V. Burke, Alessandro Chiuso, Gianluigi Pillonetto
2011 IEEE Conference on Decision and Control and European Control Conference  
We consider the problem of sparse estimation in a Bayesian framework. We outline the derivation of the Lasso in terms of marginalization of a particular Bayesian model. A different marginalization of the same model leads to a different nonconvex estimator where hyperparameters are optimized. The arguments are extended to problems where groups of variables have to be estimated. An approach alternative to Group Lasso is derived, also providing its connection with Multiple Kernel Learning. Our
more » ... mator is nonconvex but one of its versions requires optimization with respect to only one scalar variable. Theoretical arguments and numerical experiments show that the new technique obtains sparse solutions more accurate than the other two convex estimators.
doi:10.1109/cdc.2011.6160997 dblp:conf/cdc/AravkinBCP11 fatcat:ujgziube6vfvhkxp6ncgtzpjga