Guest editor's introduction

Lisa Hellerstein
1994 Machine Learning  
Research in computational learning theory covers a diverse range of topics, and the papers in this special issue reflect that diversity. There is something in these papers for (practically) everyone --foundational models, algorithms, applications, logics, and memory systems. In what follows, I will discuss the contents of these papers and their relation to broader research issues in computational learning theory, and provide references for some recent related research. Readers unfamiliar with
more » ... e field of computational learning theory may also be interested in some background reading; possibilities include three textbooks One of the central problems in computational learning theory is the development and investigation of formal learning models. The goal of a formal learning model is to capture essential properties of actual learning methods, and to enable formal analysis and the development of learning algorithms. The PAC (probably approximately correct) model of learning, introduced by Valiant (1984) , was designed to achieve that goal. The extensive research generated by the model attests to its degree of success. However, many researchers have pointed to deficiencies in the PAC model, and some have introduced variant models designed to correct these deficiencies. The first paper in this special issue, by M. J. Kearns, R. E. Schapire, and L. M. Sellie, introduces a generalization of the PAC model called the agnostic learning model. The PAC model makes strong assumptions about the form of the function being learned (the target function). However, these assumptions are not always justifiable in practice. In much of empirical machine learning research, the learning algorithm makes virtually no assumptions about the target function. The algorithm is, however, constrained to output a hypothesis from a restricted class of functions; its object is to find the "best" approximation to the target function within the hypothesis class. The agnostic learning model captures these features of empirical machine learning research, while at the same time including the PAC model as a special case. Kearns, Schapire, and Sellie prove a number of negative results showing the difficulty of agnostic learning, including an equivalence between agnostic learning and PAC learning with malicious errors (a particularly difficult noise model). They also show that agnostic learning is difficult even when the hypothesis class is the simple class of boolean conjunctions. In contrast to these negative results, they show that a number of non-trivial
doi:10.1007/bf00993467 fatcat:ww4ry7ctxfh7zd5wv7sg7n6y6a