Multivariate Adaptive Regression Splines with Non-negative Garrote Estimator

Hiroki Motogaito, Tomoyuki Sugimoto, Masashi Goto
2007 Ouyou toukeigaku  
In regression problems, some of the most important goals are (i) to obtain a lower prediction error, and (ii) to interpret regression relationships. Friedman's Multivariate Adaptive Regression Splines (MARS) method which constructs basis functions with interaction effects is a very powerful data-driven technique in the viewpoint of (i), and the single tree-based structure built by MARS contributes to approach to (ii). Also, to address (i) and (ii), the better estimation and variable selection
more » ... ariable selection of the model are inevitable issues, and then shrinkage estimators contribute to resolve such issues. Recently, especially in the context of linear regression, Breiman's non-negative garrote (NNG) estimator and Tibshirani's Lasso estimator are shown to be a stable estimation and variable selection that often outperform theirs competitors. In this paper, we focus on Breiman's NNG as a foundation to incorporate the shrinkage estimators into the tree-based regression model and propose a new version of the MARS with the NNG (NNG-MARS). Then, we evaluated some performances of the NNG-MARS via an examination of a literature example and several small scale simulations. As a conclusion, the NNG-MARS, which holds the interpretable tree-based structure, has achieved much lower prediction error than the ordinary MARS.
doi:10.5023/jappstat.36.99 fatcat:otxbo7wqkfbe5n744kqcgchsxy