Embedding Monte Carlo Search of Features in Tree-Based Ensemble Methods [chapter]

Francis Maes, Pierre Geurts, Louis Wehenkel
2012 Lecture Notes in Computer Science  
Feature generation is the problem of automatically constructing good features for a given target learning problem. While most feature generation algorithms belong either to the filter or to the wrapper approach, this paper focuses on embedded feature generation. We propose a general scheme to embed feature generation in a wide range of tree-based learning algorithms, including single decision trees, random forests and tree boosting. It is based on the formalization of feature construction as a
more » ... equential decision making problem addressed by a tractable Monte Carlo search algorithm coupled with node splitting. This leads to fast algorithms that are applicable to large-scale problems. We empirically analyze the performances of these tree-based learners combined or not with the feature generation capability on several standard datasets.
doi:10.1007/978-3-642-33460-3_18 fatcat:mfwv4fivzncazmh2f4es37jgmi