A Refined Margin Distribution Analysis for Forest Representation Learning

Shen-Huan Lyu, Liang Yang, Zhi-Hua Zhou
2019 Neural Information Processing Systems  
In this paper, we formulate the forest representation learning approach named casForest as an additive model, and show that the generalization error can be bounded by O(ln m/m), when the margin ratio related to the margin standard deviation against the margin mean is sufficiently small. This inspires us to optimize the ratio. To this end, we design a margin distribution reweighting approach for the deep forest model to attain a small margin ratio. Experiments confirm the relation between the
more » ... gin distribution and generalization performance. We remark that this study offers a novel understanding of casForest from the perspective of the margin theory and further guides the layer-by-layer forest representation learning.
dblp:conf/nips/LyuYZ19 fatcat:pev3gtbixzg33hz3mlz2mvb3ru