Online Learning and Resource-Bounded Dimension: Winnow Yields New Lower Bounds for Hard Sets [chapter]

John M. Hitchcock
2006 Lecture Notes in Computer Science  
We establish a relationship between the online mistake-bound model of learning and resourcebounded dimension. This connection is combined with the Winnow algorithm to obtain new results about the density of hard sets under adaptive reductions. This improves previous work of Fu (1995) and Lutz and Zhao (2000) , and solves one of Lutz and Mayordomo's "Twelve Problems in Resource-Bounded Measure" (1999). good mistake-bound learning algorithm. It is possible that the reduction can take
more » ... ime and that the learning algorithm can also take exponential-time, as long as the mistake bound of the algorithm is subexponential. If we have a reduction from the unknown set to a concept in learnable concept class, we can view the reduction as generating a sequence of examples, apply the learning algorithm to these examples, and use the learning algorithm's predictions to design a good betting strategy. Formal details of this framework are given in Section 3. Density of Hard Sets The two most common notions of polynomial-time reductions are many-one (≤ p m ) and Turing (≤ p T ). A many-one reduction from A to B maps instances of A to instance of B, preserving membership. A Turing reduction from A to B makes many, possibly adaptive, queries to B in order to solve A. Many-one reductions are a special case of Turing reductions. In between ≤ p m and ≤ p T is a wide variety of polynomial-time reductions of different strengths. A common use of reductions is to demonstrate hardness for a complexity class. Let ≤ p τ be a polynomial-time reducibility. For any set B, let P τ (B) = {A | A ≤ p τ B} be the class of all problems that ≤ p τ -reduce to B. We say that B is ≤ p τ -hard for a complexity class C if C ⊆ P τ (B), that is, every problem in C ≤ p τ -reduces to B. For a class D of sets, a useful notation is P τ (D) = B∈D P τ (B). A problem B is dense if there exists > 0 such that |B ≤n | > 2 n for all but finitely many n. All known hard sets for the exponential-time complexity classes E = DTIME(2 O(n) ) or EXP = DTIME(2 n O(1) ) are dense. Whether every hard set must be dense has been often studied. First, Meyer [25] showed that every ≤ p m -hard set for E must be dense, and he observed that proving the same for ≤ p T -reductions would imply that E has exponential circuit-size complexity. Since then, a line of research has obtained results for a variety of reductions between ≤ p m and ≤ p T , specifically the conjunctive (≤ p c ) and disjunctive (≤ p d ) reductions, and for various functions f (n), the bounded query ≤ p f (n)−tt and ≤ p f (n)−T reductions: 1. Watanabe [27, 10] showed that every hard set for E under the ≤ p c , ≤ p d , or ≤ p O(log n)−tt reductions is dense. [20] showed that for all α < 1, the class P n α −tt (DENSE c ) has p-measure 0, where DENSE is the class of all dense sets. Since E does not have p-measure 0, their result implies that every ≤ p n α −tt -hard set for E is dense. Lutz and Mayordomo 3. Fu [8] showed that for all α < 1/2, every ≤ p n α −T -hard set for E is dense, and that for all α < 1, every ≤ p n α −T -hard set for EXP is dense. Lutz and Zhao [22] gave a measure-theoretic strengthening of Fu's results, showing that for all α < 1/2, P n α −T (DENSE c ) has p-measure 0, and that for all α < 1, P n α −T (DENSE c ) has p 2 -measure 0. This contrast between E and EXP in the last two references was left as a curious open problem, and exposited by Lutz and Mayordomo [21] as one of their "Twelve Problems in Resource-Bounded Measure": Problem 6. For α ≤ 1 2 < 1, is it the case that P n α −T (DENSE c ) has p-measure 0 (or at least, that E ⊆ P n α −T (SPARSE))? We resolve this problem, showing the much stronger conclusion that the classes in question have p-dimension 0. But first, in Section 4, we prove a theorem about disjunctive reductions that ftpmail@ftp.eccc.uni-trier.de, subject 'help eccc' ftp://ftp.eccc.uni-trier.de/pub/eccc http://www.eccc.uni-trier.de/eccc ECCC
doi:10.1007/11672142_33 fatcat:v47l6siel5haxih6ehriiljgx4