### In Praise of Strategies [chapter]

Johan van Benthem
2012 Lecture Notes in Computer Science
This note high-lights one major theme in my lecture notes Logic in Games (van Benthem 1999 -2002 : the need for explicit logics that define agents' strategies, as the drivers of interaction in games. Our text outlines issues, recalls published results from the last few years, and raises new open problems. Results are mainly quoted, and the mephasis is on new notions and open problems. For more details on the various topics discussed, see the relevant references. Strategies as first-class
more » ... s Much of game theory is about the question whether strategic equilibria exist. But there are hardly any explicit languages for defining, comparing, or combining strategies as such -the way we have them for actions and plans, maybe the closest intuitive analogue to strategies. True, there are many current logics for describing game structure -but these tend to have existential quantifiers saying that "players have a strategy" for achieving some purpose, while descriptions of these strategies themselves are not part of the logical language (cf. Parikh & Pauly 2003, van der Hoek, van Otterloo & Wooldridge 2005). Therefore, I consider strategies 'the unsung heroes of game theory' -and I want to show how the right kind of logic can bring them to the fore. One guide-line of adequacy for doing so, in the fastgrowing jungle of 'game logics', is the following: we would like to explicitly represent the elementary reasoning about strategies underlying many basic game-theoretic results. Or in more general terms, we want to explicitly represent agents' reasoning about their plans. 2 Games as models for modal process logics Van Benthem 2002, mainly an extract from the lecture notes Logic in Games, shows how modal logic fits naturally with extensive games, viewed as process models from computer science, viz. labeled transition systems with some special annotation for players' activities. Basic modal logic Extensive game trees may be viewed as state spaces for some multi-agent process. Labeled modalities f then express that some move a is available leading to a next node in the game tree satisfying f. Then modal operator combinations describe potential interaction. For instance, in a self-explanatory notation, the formula [move-A]f says that, at the current node of evaluation, player E has a strategy for responding to A's initial move which ensures that f results after two steps of play. Extending this to extensive games up to some finite depth k, and using alternations []<>[]<>... of modal operators up to length k to reach the end-points of the game tree, we can express the existence of winning strategies and the like in fixed finite games. Indeed, given this connection, with finite depth, standard logical laws have immediate game-theoretic import. In particular, consider the valid law of excluded middle in the following modal form []<>[]<>...f ⁄ <>[]<>[]...¬f, where the dots indicate the depth of the tree. This expresses the determinacy of these games, as stated in Zermelo's Theorem that all finite zero-sum two-player games are determined. Modal m-calculus Unfortunately, such modal game-by-game definitions are not 'generic', as they depend on the particular model considered -and Zermelo's inductive argument is rendered much more faithfully by means of just one fixed formula in the modal m-calculus (Bradfield & Stirling 2006). To make the relevant point more generally, let us first define the following 'forcing modality'in games: M, s |= {i}f iff player i has a strategy for the sub-game starting at s which guarantees that only nodes will be visited where f holds, whatever the other does. Forcing talk is widespread in games, and it is an obvious target for logical formalization. Note that, for convenience, {i}f talks about intermediate nodes, not just end nodes of the game. The existence of a winning strategy for player i can now be expressed as {i} (end AE win i ) 1 Here is an explicit definition for this assertion in the modal m-calculus, using some further obvious proposition letters and action symbols for indicating players' turns and moves at nodes of the game tree. In this formula, the symbol j is used fort the other player in the game: This definition is faithful to the obvious recursive meaning of having a strategy for player i, regardless of what the others do -and it is generic, since it works in all games viewed as models M for our language. Incidentally, we use a greatest fixed-point operator nq• here, rather than a smallest fixed-point operator mq•, for easier extension to infinite games later on. Propositional dynamic logic But now to strategies as such! An obvious candidate for defining these is propositional dynamic logic PDL, combining propositions about nodes in the game tree with programs defining transition relations between such nodes. Without loss of information, a strategy for player i is a binary relation between nodes. At turns for player i, 1 Here, end is a proposition letter for end nodes, or a complex modal formula saying that no move is possible, and win i is a proposition letter saying that player i wins at the current node. 3 it picks out one transition, at turns for the others, it allows any move. Here is a simple fact for a start. General PDL-programs now define strategies s. Note that, in general, these programs may denote arbitrary transition relations, not just uniquely valued functions. Such relations constrain i's moves at her turns, without necessarily narrowing them down to just a single one. This possible non-determinacy is fine, since it makes eminent sense for plans, and by extension, also for a broader notion of strategies in games. Now, we can define an explicit version of the earlier forcing modality, indicating the strategy involved -even without recourse to the full power of the modal m-calculus: Fact 1 For any program expression s, PDL can define the explicit forcing modality {s, i}f stating that s is a strategy for player i forcing the game, against any play of the others, to pass only through states satisfying f. The precise definition is an easy exercise in modal logic (cf. van Benthem 2002). But PDL has further uses in this setting. Consider any finite game M with a strategy s for player i. As a relation, s is a finite set of ordered pairs (s, t). Thus, it can be defined by enumeration as a program union, if we define these ordered pairs. To do so, assume we have an 'expressive' model M, where states s are definable in our modal language by formulas def s . 2 Then we define transitions (s, t) by formulas def s ; a; def t , with a the relevant move: Fact 2 In expressive finite extensive games, all strategies are PDL-definable. Of course, this is a trivial result, but it does suggest that PDL is on the right track. Van Benthem 2002 also discusses further issues about PDL as a 'calculus of strategies'. For instance, suppose that player E plays strategy s, and at the same time, A plays strategy t. What end nodes are reachable in this way? (With standard strategies, a unique outcome will be reached.) This calls for an operation on strategies describing the joint strategy of {E, A}and with a little reflection, it is clear that this joint strategy is just the intersection s«t of the relations s, t. This operation takes us outside of PDL proper, but then, PDL « with intersections added is still a reasonably simple modal language. Stronger modal logics of strategies? The modal m-calculus is a natural strengthening of PDL, but it has no explicit programs or strategies, as its formulas merely define properties of states. Is there a counterpart to the m--calculus which also extends PDL in terms of defining corresponding transition relations? E.g., a strategy 'keep playing a' guarantees infinite abranches for true greatest fixed-point formulas like np• p. 3 3 Preference structure and more realistic games Real games add preferences for players over outcome states, or utility values beyond 'win' and 'lose'. In this case, defining the so-called Backward Induction procedure for solving extensive games, rather than just Zermelo winning positions, becomes a benchmark for game logics. Here the issue is not whetherf this can be defined at all. Any simple game concept can be phrased in some modal-like language with transition relations for moves, provided one adds suitable modalities for the preference order. But can the paraphrase be done in a perspicuous manner, generating some new insight? Fact 3 The Backward Induction path is definable in modal preference logic. Solutions have been published by Board, Bonanno, and many others: cf. Harrenstein 2004, De Bruin 2004. Given all this, we do not state an explicit PDL-style solution here. Betterness, preference, and expectation Actually, the situation to be analyzed is somewhat subtle conceptually. A game gives players' direct preferences over outcomes. The Backward Induction algorithm then lifts these to a binary order among intermediate nodes. But as pointed out in van Benthem 2002, 2007D, this order then gets a re-interpretation. It does not just represent what players prefer, but what they expect to happen, given rationality assumptions about how the other players will proceed. Thus, the resulting binary order is more like the plausibility relations used to interpret conditional beliefs in doxastic logic, generated from a mixture of preference and assumptions about behaviour of other players. Modal preference languages come in many kinds. Some recent proposals are in Harrenstein makes Backward Induction tick, using a preference modality f : player i prefers some node where f holds to the current one. It then defines the backward induction path as a unique relation s, not by a modal formula over models M, but via the following frame correspondence on finite structures: 3 Van Benthem 2005A looks at richer fragments than PDL with programs as solutions to fixed-point equations of special forms,guaranteeing uniform convergence by stage w. Fact 4 The BI strategy is definable as the unique relation s satisfying the following axiom for all propositions P -viewed as sets of nodes -, for all players i: