Learning from small data sets is difficult in the absence of specific domain knowledge. We present a regularized linear model called STEW, which benefits from a generic and prevalent form of prior knowledge: feature directions. STEW shrinks weights toward each other, converging to an equalweights solution in the limit of infinite regularization. We provide theoretical results on the equalweights solution that explains how STEW can productively trade-off bias and variance. Across a wide range of

more »

... learning problems, including Tetris, STEW outperformed existing linear models, including ridge regression, the Lasso, and the non-negative Lasso, when feature directions were known. The model proved to be robust to unreliable (or absent) feature directions, outperforming alternative models under diverse conditions. Our results in Tetris were obtained by using a novel approach to learning in sequential decision environments based on multinomial logistic regression.