QBF-Based Formal Verification: Experience and Perspectives

Marco Benedetti, Hratch Mangassarian, Miroslav N. Velev
2008 Journal on Satisfiability, Boolean Modeling and Computation  
The language of Quantified Boolean Formulas (QBF) has a lot of potential applications to Formal Verification (FV) tasks, as it captures many of these tasks in a natural and compact way. Practical experience has been disappointing though. When compared with contending approaches such as SAT, QBF-based FV has invariably yielded unfavorable experimental results. This paper makes two contributions. We first provide an account of the status quo in QBF-based FV. We examine commonly adopted
more » ... ons and the relative strengths of different decision procedures. In the second part of this paper, we investigate for the first time the relevance of some advanced QBF techniques to FV tasks. In particular, we describe the use and the benefits of restricted quantifiers, QBF certificates, alternative encodings for classical model checking problems, and encodings with free variables. These promising research perspectives seem to reverse the negative standing of QBF applied to FV, as confirmed by the experimental evidence we discuss. Experiments are conducted by extending the publicly available solver sKizzo in several ways, and they include the first case studies where QBF compares favorably to SAT, its traditional competitor. QBF turns out to be an order of magnitude faster than SAT in some tasks (e.g., automated design debugging of large circuits). Moreover, as the size of the problems grows, the SAT encodings result in excessive memory requirements leading to out-of-memory conditions, while the more compact QBF encodings continue to be manageable and solvable. Satisfiability (SAT) solvers have been successfully used to address a large class of industrialscale problems [33] in the area of computer-aided design of integrated circuits [48, 50] and Model Checking for dynamic systems [21] , to name a few. On the other hand, the more expressive language of QBF, which adds the valuable possibility to quantify-universally or existentially-over the truth value of each variable, captures the wider class of PSPACE-complete problems. This enables us to produce expressive and compact formulations of many formal verification tasks that would require a significantly larger description in PROP. But, do QBF solvers add substantial value to the reasoning capabilities of SAT solvers as far as FV tasks are concerned? More broadly speaking, are they ready to become the reference technology in any class of FV tasks? Many results suggest that this is not yet the case [75, 59, 46, 45] . So far, solutions to FV problems based on propositional logic have seen their best embodiment in procedures relying on SAT solvers. Researchers have been confronted by heavy time/memory tradeoffs in their attempts to shift to more powerful formalisms, such as the QBF logic we consider here. For example, despite the ability of QBF to capture FV problems in "compressed" forms, these shorter formalizations have turned out to be more time-intensive to deal with than SAT-based ones. It has not been fully understood if such a time/memory trade-off is inescapable, as some believe, or whether it can be bypassed in practical cases. It has also been unclear whether the problem can be resolved by improving existing solvers or by alternative decision procedures, or whether the issue is more about the way QBF encodings of FV problems are formulated than about the solving strategy. This paper contributes to the above open research questions in two ways: (1) We provide the first thorough FV-oriented survey of the current state-of-the-art in QBF solvers, benchmarks, and encodings, and (2) we bring in concepts and findings that recently emerged in the QBF and related research communities, and we start to investigate their potential in FV. The paper is accordingly divided into two parts: The Experience part (Section 2) looks at the current situation. After a basic introduction to QBF (Section 2.1), we report on the status quo of QBF-based formal verification in terms of commonly adopted formalizations (Section 2.2), established benchmarks in the public domain (Section 2.3), existing QBF decision procedures (Section 2.4) and their relative strengths compared to each other (Section 2.5) and compared to alternative SAT-based approaches (Section 2.6). Some new insights on how challenging instances are solved by state-of-the-art provers are given in Section 2.5. The Perspectives part (Section 3) looks at the future. We introduce promising research directions which might reverse the current reputation of QBF as a technology unsuited to real-world applications. Recent results from related research communities are reinterpreted for QBF and from the perspective of FV applications, namely the restricted quantification technique (Section 3.1) recently introduced [19] to model and solve quantified constraint satisfaction problems (QCSPs), and the alternative encodings for bounded sequential modeling (Section 3.2) proposed by the design automation community [55, 56] . The relevance to FV of new QBF-specific techniques, such as validity certification [13] (Section 3.3) and encodings with free variables (Section 3.4), are investigated for the first time. We use the 1. the first carry is false: ¬r 0 ;
doi:10.3233/sat190055 fatcat:smzdgdfttzgmjcigbvfhwlz5wm