Agnostic Boosting [chapter]

Shai Ben-David, Philip M. Long, Yishay Mansour
2001 Lecture Notes in Computer Science  
We extend the boosting paradigm to the realistic setting of agnostic learning, that is, to a setting where the training sample is generated by an arbitrary (unknown) probability distribution over examples and labels. We de ne a -weak agnostic learner with respect to a hypothesis class F as follows: given a distribution P it outputs some hypothesis h 2 F whose error is at most erP(F) + , where erP (F) is the minimal error of an hypothesis from F under the distribution P (note that for some
more » ... butions the bound may exceed a half). We show a boosting algorithm that using the weak agnostic learner computes a hypothesis whose error is at most maxfc1( )er(F) c 2 ( ) ; g, in time polynomial in 1= . While this generalization guarantee is significantly weaker than the one resulting from the known PAC boosting algorithms, one should note that the assumption required for -weak agnostic learner is much weaker. In fact, an important virtue of the notion of weak agnostic learning is that in many cases such learning is achieved by e cient algorithms.
doi:10.1007/3-540-44581-1_33 fatcat:wc7x7pes6fcjlaldhpwajdtsae