A novel technique is proposed to construct sparse regression models based on the orthogonal least squares method with tunable kernels. The proposed technique tunes the centre vector and diagonal covariance matrix of individual regressor by incrementally minimising the training mean square error using a guided random search algorithm, and it offers a state-of-theart method for constructing very sparse models that generalise well.