Superfast-Trainable Multi-Class Probabilistic Classifier by Least-Squares Posterior Fitting

Masashi SUGIYAMA
2010 IEICE transactions on information and systems  
Kernel logistic regression (KLR) is a powerful and flexible classification algorithm, which possesses an ability to provide the confidence of class prediction. However, its training-typically carried out by (quasi-)Newton methods-is rather time-consuming. In this paper, we propose an alternative probabilistic classification algorithm called Least-Squares Probabilistic Classifier (LSPC). KLR models the class-posterior probability by the log-linear combination of kernel functions and its
more » ... s are learned by (regularized) maximum likelihood. In contrast, LSPC employs the linear combination of kernel functions and its parameters are learned by regularized least-squares fitting of the true class-posterior probability. Thanks to this linear regularized least-squares formulation, the solution of LSPC can be computed analytically just by solving a regularized system of linear equations in a class-wise manner. Thus LSPC is computationally very efficient and numerically stable. Through experiments, we show that the computation time of LSPC is faster than that of KLR by two orders of magnitude, with comparable classification accuracy. key words: probabilistic classification, kernel logistic regression, classposterior probability, squared-loss * A least-squares formulation has been employed for improving the computational efficiency of SVMs [13], [28], [31] . However, these approaches deal with deterministic classification, not probabilistic classification.
doi:10.1587/transinf.e93.d.2690 fatcat:ywnz7kdisndjrkvr4iqep37yli