Parsimonious least squares support vector regression using orthogonal forward selection with the generalised kernel model

Xunxian Wang, David Lowe, Sheng Chen, Chris J. Harris
2006 International journal of Modeling, identification and control  
A sparse regression modelling technique is developed using a generalised kernel model in which each kernel regressor has its individually tuned position (centre) vector and diagonal covariance matrix. An orthogonal least squares forward selection procedure is employed to append the regressors one by one. After the determination of the model structure, namely the selection of an appropriate number of regressors, the model weight parameters are calculated from the Lagrange dual problem of the
more » ... problem of the original least squares problem. Different from the least squares support vector regression, this regression modelling procedure involves neither reproducing kernel Hilbert space nor Mercer decomposition concepts. As the regressors used are not restricted to be positioned at training input points and each regressor has its own diagonal covariance matrix, a very sparse representation can be obtained with excellent generalisation capability. Experimental results involving two real data sets demonstrate the effectiveness of the proposed regression modelling approach. (2006) 'Parsimonious least squares support vector regression using orthogonal forward selection with the generalised kernel model', Int. in September 1999. Professor Chen's research interests include wireless communications, machine learning, finite-precision digital controller design and evolutionary computation. David Lowe has held the Chair of Neural Computing at Aston University, UK, since 1994. He is a Coinventor of the Radial Basis Function neural network architecture. His current research activities relate to stochastic generative control, biomedical applications of statistical pattern processing focussing on DNA microarrays and EEG/MEG brain signal analysis and non-linear methods for digital steganography.
doi:10.1504/ijmic.2006.012612 fatcat:442rzyvd3bbahbtbpypjqu6fza