Robustness and Tractability for Non-convex M-estimators

Ruizhi Zhang, Yajun Mei, Jianjun Shi, Huan Xu
2022 Statistica sinica  
We investigate two important properties of M-estimators, namely, robustness and tractability, in the linear regression setting, when the observations are contaminated by some arbitrary outliers. Specifically, robustness means the statistical property that the estimator should always be close to the true underlying parameters regardless of the distribution of the outliers, and tractability indicates the computational property that the estimator can be computed efficiently, even if the objective
more » ... n if the objective function of the M-estimator is non-convex. In this article, by learning the landscape of the empirical risk, we show that under some sufficient conditions, many M-estimators enjoy nice robustness and tractability properties simultaneously when the percentage of outliers is small. We further extend our analysis to the high-dimensional setting, where the number of parameters is greater than the number of samples, p n, and prove that when the proportion of outliers is small, the penalized M-estimators with L1 penalty will enjoy robustness and tractability simultaneously. Our research provides an analytic approach to see the effects of outliers and tuning parameters on the robustness and tractability of some families of M-estimators. Simulation and case studies are presented to illustrate the usefulness of our theoretical results for Statistica Sinica: Newly accepted Paper (accepted author-version subject to English editing) M-estimators under Welsch's exponential squared loss and Tukey's bisquare loss.
doi:10.5705/ss.202019.0324 fatcat:nc2gslz5srhkpcuavilyvzvvxq