Quasipolynomial-Time Identity Testing of Non-commutative and Read-Once Oblivious Algebraic Branching Programs

Michael A. Forbes, Amir Shpilka
2013 2013 IEEE 54th Annual Symposium on Foundations of Computer Science  
We study the problem of obtaining efficient, deterministic, black-box polynomial identity testing algorithms (PIT) for read-once oblivious algebraic branching programs (ABPs). This class has an efficient, deterministic, white-box polynomial identity testing algorithm (due to Raz and Shpilka [RS05]), but prior to this work had no known such black-box algorithm. Here we obtain the first quasi-polynomial sized hitting sets for this class, when the order of the variables is known. This work can be
more » ... een as an algebraic analogue of the results of Nisan [Nis92] and Impagliazzo-Nisan-Wigderson [INW94] for space-bounded pseudorandom generators. We also show that several other circuit classes can be black-box reduced to read-once oblivious ABPs, including set-multilinear ABPs (a generalization of depth 3 set-multilinear formulas), non-commutative ABPs (generalizing non-commutative formulas), and (semi-)diagonal depth-4 circuits (as introduced by Saxena [Sax08], and recently shown by Mulmuley [Mul12] to have implications for derandomizing Noether's Normalization Lemma). For set-multilinear ABPs and non-commutative ABPs, we give quasi-polynomial-time black-box PIT algorithms, where the latter case involves evaluations over the algebra of (D + 1) × (D + 1) matrices, where D is the depth of the ABP. For (semi-)diagonal depth-4 circuits, we obtain a black-box PIT algorithm (over any characteristic) whose run-time is quasi-polynomial in the runtime of Saxena's whitebox algorithm, matching the concurrent work of Agrawal, Saha, and Saxena [ASS12]. Finally, by combining our results with the reconstruction algorithm of Klivans and Shpilka [KS06], we obtain deterministic reconstruction algorithms for the above circuit classes. * ≡ 0". An important open problem is to find a derandomization of this algorithm, that is, to find a deterministic procedure for PIT that runs in polynomial time (in the size of the circuit C). One interesting property of the above randomized algorithm of Schwartz-Zippel is that the algorithm does not need to "see" the circuit C. Namely, the algorithm only uses the circuit to compute the evaluations f (α 1 , . . . , α n ). Such an algorithm is called a black-box algorithm. In contrast, an algorithm that can access the internal structure of the circuit C is called a white-box algorithm. Clearly, the designer of the algorithm has more resources in the white-box model and so one can expect that solving PIT in this model should be a simpler task than in the black-box model. The problem of derandomizing PIT has received a lot of attention in the past few years. In particular, many works examine a particular class of circuits C, and design PIT algorithms only for circuits in that class. One reason for this attention is the strong connection between deterministic PIT algorithms for a class C and lower bounds for C. This connection was first observed by Heintz and Schnorr [HS80] (and also by Agrawal [Agr05]) for the black-box model and by Kabanets and Impagliazzo [KI04] for the white-box model (see also Dvir, Shpilka and Yehudayoff [DSY09]). Another motivation for studying the problem is its relation to algorithmic questions. Indeed, the famous deterministic primality testing algorithm of Agrawal, Kayal and Saxena [AKS04] is based on derandomizing a specific polynomial identity. Finally, the PIT problem is, in some sense, the most general problem that we know today for which we have randomized coRP algorithms but no polynomial time algorithms, thus studying it is a natural step towards a better understanding of the relation between RP and P. For more on the PIT problem we refer to the survey by Shpilka and Yehudayoff [SY10] . Although the white-box model seems to be simpler than the black-box model, for most models for which a white-box PIT algorithm is known, also a black-box PIT algorithm is known, albeit sometimes with worse parameters. Such examples include depth-2 circuits (also known as sparse polynomials) [BOT88, KS01], depth-3 ΣΠΣ(k) circuits [SS11], Read-k formulas [AvMV11] and depth-3 tensors (also known as depth-3 set-multilinear circuits) [RS05, FS12] . While the running time of the algorithms for depth-2 circuits and ΣΠΣ(k) circuits are essentially the same in the whitebox and black-box models, for Read-k formulas and set-multilinear depth-3 circuits we encounter some loss that results in a quasi-polynomial running time in the black-box model compared to a polynomial time algorithm in the white-box model. Until this work, the only model for which an efficient white-box algorithm was known without a (sub-exponential) black-box counterpart was the model of non-commutative arithmetic formulas, or, more generally, the models of non-commutative algebraic branching programs (ABPs) and setmultilinear algebraic branching programs [RS05] (see Subsection 1.1 for definitions). The main result in this paper is a quasi-polynomial time PIT algorithm in the black-box model for read-once oblivious algebraic branching programs. Using this result we obtain black-box algorithms of similar running times for the models of set-multilinear ABPs, non-commutative ABPs, as well as for diagonal circuits as defined by Saxena [Sax08]. Although exponential lower bounds are known for these models, we note that the algebraic hardness-versus-randomness result of Kabanets and Impagliazzo [KI04] (as well as the extension of this result by Dvir, Shpilka and Yehudayoff [DSY09] to low-depth circuits) does not imply a black-box PIT algorithm for the model, since their technique does not work for the restricted models studied here. Although the models which are considered here may seem a bit unnatural at first sight, we note that the non-commutative model received a lot of attention in the algebraic complexity literature (mostly in the last few years) as it gives a good starting point for understanding the hardness of computing the determinant and permanent [Nis91, CS07, AJS09, AS10, HWY10a, CHSS11,
doi:10.1109/focs.2013.34 dblp:conf/focs/ForbesS13 fatcat:m6tsod2pnjcatlahg5zzke2b4a