Tight Bounds on ℓ 1 Approximation and Learning of Self-Bounding Functions

Vitaly Feldman, Pravesh Kothari, Jan Vondrák
2017 International Conference on Algorithmic Learning Theory  
We study the complexity of learning and approximation of self-bounding functions over the uniform distribution on the Boolean hypercube {0, 1} n . Informally, a function f : {0, 1} n → R is self-bounding if for every x ∈ {0, 1} n , f (x) upper bounds the sum of all the n marginal decreases in the value of the function at x. Self-bounding functions include such wellknown classes of functions as submodular and fractionally-subadditive (XOS) functions. They were introduced by Boucheron et al. in
more » ... e context of concentration of measure inequalities. Our main result is a nearly tight 1 -approximation of self-bounding functions by low-degree juntas. Specifically, all self-bounding functions can be -approximated in 1 by a polynomial of degree Õ(1/ ) over 2 Õ(1/ ) variables. We show that both the degree and junta-size are optimal up to logarithmic terms. Previous techniques considered stronger 2 approximation and proved nearly tight bounds of Θ(1/ 2 ) on the degree and 2 Θ(1/ 2 ) on the number of variables. Our bounds rely on the analysis of noise stability of self-bounding functions together with a stronger connection between noise stability and 1 approximation by low-degree polynomials. This technique can also be used to get tighter bounds on 1 approximation by low-degree polynomials and faster learning algorithm for halfspaces. These results lead to improved and in several cases almost tight bounds for PAC and agnostic learning of self-bounding functions relative to the uniform distribution. In particular, assuming hardness of learning juntas, we show that PAC and agnostic learning of self-bounding functions have complexity of n Θ(1/ ) .
dblp:conf/alt/FeldmanKV17 fatcat:veyihsja3vcxnmyxvhju5o7tdy