An efficient extension to mixture techniques for prediction and decision trees

Fernando Pereira, Yoram Singer
1997 Proceedings of the tenth annual conference on Computational learning theory - COLT '97  
We present an efficient method for maintaining mixtures of prunings of a prediction or decision tree that extends the previous methods for "node-based" prunings () to the larger class of edge-based prunings. The method includes an online weight-allocation algorithm that can be used for prediction, compression and classification. Although the set of edge-based prunings of a given tree is much larger than that of node-based prunings, our algorithm has similar space and time complexity to that of
more » ... revious mixture algorithms for trees. Using the general online framework of Freund & Schapire (1997), we prove that our algorithm maintains correctly the mixture weights for edge-based prunings with any bounded loss function. We also give a similar algorithm for the logarithmic loss function with a corresponding weight-allocation algorithm. Finally, we describe experiments comparing node-based and edge-based mixture models for estimating the probability of the next word in English text, which show the advantages of edge-based models.
doi:10.1145/267460.267487 dblp:conf/colt/PereiraS97 fatcat:tyl3mtpluratzgoejrbuhv7cqa