Rationality for subclasses of 321-avoiding permutations
European journal of combinatorics (Print)
3 We prove that every proper subclass of the 321-avoiding permutations 4 that is defined either by only finitely many additional restrictions or is 5 well-quasi-ordered has a rational generating function. To do so we show 6 that any such class is in bijective correspondence with a regular language. 7 The proof makes significant use of formal languages and of a host of 8 encodings, including a new mapping called the panel encoding that maps 9 languages over the infinite alphabet of positive
... ers avoiding certain 10 subwords to languages over finite alphabets. 11 It has been known since 1968, when the first volume of Knuth's The Art of Computer Program-13 ming  was published, that the 312-avoiding permutations and the 321-avoiding permutations 14 are both enumerated by the Catalan numbers, and thus have algebraic generating functions. At 15 least nine essentially different bijections between these two permutation classes have been devised 16 in the intervening years, as surveyed by Claesson and Kitaev . In one such bijection (shown 17 in Figure 1 and first given in this non-recursive form by Krattenthaler ) we obtain Dyck paths 18 from permutations of both types by drawing a path above their left-to-right maxima (an entry is a 19 left-to-right maximum if it is greater than every entry to its left). 1 12 21 123 132 213 231 321 1 12 21 123 132 213 231 312 Figure 2: The Hasse diagrams of 312-avoiding (left) and 321-avoiding (right) permutations. Despite their equinumerosity, there are fundamental differences between these two classes. Indeed, 21 Miner and Pak  make a compelling argument that there are so many different bijections between 22 these two classes precisely because they are so different, and thus there can be no "ultimate" bijection. 23 In particular, both sets carry a natural ordering with respect to the containment of permutations 24 (defined below) but they are not isomorphic as partially ordered sets. Indeed, this can be seen by 25 examining the first three levels of their Hasse diagrams, drawn in Figure 2 . 26 A more striking difference between the two classes is that the 321-avoiding permutations contain in-27 finite antichains (see Section 9), while the 312-avoiding permutations do not. Following the standard 28 terminology, we say that a permutation class without infinite antichains is well-quasi-ordered. 29 From a structural perspective, the avoidance of 312 imposes severe restrictions on permutations: the 30 entries to the left of the minimum must lie below the entries to the right of this minimum. This 31 restricted structure is known to imply that proper subclasses of the 312-avoiding permutations are 32 very well-behaved: there are only countably many such subclasses, and as Albert and Atkinson  33 proved in their work on the substitution decomposition, each has a rational generating function. 34 (Mansour and Vainshtein  had proved this rationality result for proper subclasses classes defined 35 by a single additional restriction earlier.) 36 The 321-avoiding permutations also have a good deal of structure: their entries can be partitioned 37 into two increasing subsequences. However, this property has proved much more difficult to work 38 with. In particular, as noted above, there are infinite antichains of 321-avoiding permutations, so 39 there are uncountably many proper subclasses of this class-in fact uncountably many subclasses 40 with pairwise distinct generating functions. By an elementary counting argument, some of these 41 proper subclasses must have non-rational (indeed, also non-algebraic and non-D-finite) generating 42 functions. 43 Because Av(321) is not well-quasi-ordered, any result analogous to the one mentioned for 312-44 avoiding permutations (which are, to repeat, well-quasi-ordered) must be more discerning as to 52 positive integers. We define the length of the permutation π, denoted |π|, to be the length of the 53 corresponding sequence, i.e., the cardinality of the domain of π. Given permutations π and σ, we say 54 that π contains σ, and write σ ≤ π, if π has a subsequence π(i 1 ) · · · π(i |σ| ) of the same length as σ that 55 is order isomorphic to σ (i.e., π(i s ) < π(i t ) if and only if σ(s) < σ(t) for all 1 ≤ s, t ≤ |σ|); otherwise, 56 we say that π avoids σ. If π contains σ we also say that σ is a subpermutation of π particularly in 57 contexts where we have a specific embedding (i.e., set of indices) in mind. Containment is a partial 58 order on permutations. For example, π = 251634 contains σ = 4123, as can be seen by considering 59 the subsequence π(2)π(3)π(5)π(6) = 5134. A collection of permutations C is a permutation class if 60 it is closed downwards in this order; i.e., if π ∈ C and σ ≤ π, then σ ∈ C. 61 For any permutation class C there is a unique antichain B such that This antichain, consisting of the minimal permutations not in C, is called the basis of C. If B happens 64 to be finite, we say that C is finitely based. For non-negative integers n, we denote by C n the set of 65 permutations in C of length n, and refer to 66 n |C n |x n = π∈C x |π| 67 as the generating function of C. The goal of this paper is to establish the following. 68 Theorem 1.1. If a proper subclass of the 321-avoiding permutations is finitely based or well-quasi-69 ordered then it has a rational generating function. 70 In  Bousquet-Mélou writes 71 "for almost all families of combinatorial objects with a rational [generating function], it 72 is easy to foresee that there will be a bijection between these objects and words of a 73 regular language". 74 In proving Theorem 1.1 we indeed adopt an approach via regular languages. We in fact encode 75 permutations as words using several different encodings. We begin by introducing the domino 76 encoding that records the relative positions of entries in pairs of adjacent cells in a staircase gridding. 77 After that we combine this information and encode each 321-avoiding permutation as a word, say 78 w, over the positive integers P satisfying the additional condition w(i + 1) ≤ w(i) + 1 for all relevant 79 indices i (throughout we denote by w(i) the i th letter of the word w). We then show that for 80 any proper subclass, C, of 321-avoiding permutations there is some positive integer c such that the 81 encoding of every permutation in C avoids (as a subword) every shift of the word (12 · · · c) c , i.e. all 82 words (i(i + 1) · · · (i + c − 1)) c for i ∈ P. The true key to our method is the panel encoding η c , which translates languages not containing shifts of (12 · · · c) c to languages over finite alphabets. A careful 84 analysis of the interplay between panel encodings, domino encodings, and the classical encodings 85 by Dyck paths (from Figure 1) along with a technique called marking establishes the regularity of 86 various images under η c , and this completes the proof of Theorem 1.1. 87 We assume throughout that the reader has some familiarity with the basics of regular languages, as 88 provided by Sakarovitch ; for a more combinatorial approach we refer the reader to Bousquet-89 Mélou  or Flajolet and Sedgewick [16, Section I.4 and Appendix A.7]. The notation used is 90 mostly standard. Herein a subword of the word w is any subsequence of its entries while a factor is 91 a contiguous subsequence. Given a set of letters X and a word w we denote by w| X the projection 92 of w onto X, i.e., the subword of w formed by its letters in X. Finally, we denote the empty word 93 by . 94 2. Staircase Griddings 95 A staircase gridding of a 321-avoiding permutation π is a partition of its entries into cells labelled 96 by the positive integers satisfying four properties: 97 • the entries in each cell are increasing, 98 • for i ≥ 1, all entries in the (2i) th cell lie to the right of those in the (2i − 1) st cell, 99 • for i ≥ 1, all entries in the (2i + 1) st cell lie above those in the (2i) th cell, and 100 • if j ≥ i + 2 then all entries in the j th cell lie above and to the right of those in the i th cell. 101 Staircase griddings have been used extensively in the study of 321-avoiding permutations, for instance 102 in [3, 7, 9, 17] and represent the fundamental objects of consideration here. We denote by π a 103 particular staircase gridding of the 321-avoiding permutation π. 104 Every 321-avoiding permutation has at least one staircase gridding and indeed, we can identify 105 a preferred staircase gridding of every such permutation: a staircase gridding of the 321-avoiding 106 permutation π is greedy if the first cell contains as many entries as possible, and subject to this, 107 the second cell contains as many entries as possible, and so on. Figure 3 provides an example of a 108 greedy staircase gridding. 109 It is easy to construct greedy staircase griddings in the following iterative manner. The entries 110 of the first cell are the maximum increasing prefix τ of π. Those of the second cell are then the 111 maximum increasing sequence in π \τ whose values form an initial segment of the values occurring in 112 π \ τ . Thereafter we continue alternately taking a maximum increasing prefix and then a maximum 113 increasing sequence of values forming an initial segment of the values remaining. 114 The relative position of two entries in a 321-avoiding permutation π is completely determined by 115 the numbers given to their cells in any staircase gridding, unless these numbers are consecutive. In 116 the case of cells which lie next to each other horizontally we consider their entries as being ordered 117 from bottom to top, and in the case of cells which lie next to each other vertically, from left to right. 118 Observe that this gives us two orders on the entries of a given cell (except the first), but that the two 119 orders in fact coincide. With this ordering in mind, we formulate two conditions that characterise 120 greedy staircase griddings: 126 Proposition 2.1. A staircase gridding is greedy if and only if it satisfies (G1) and (G2). 127 Proof. Let π be a 321-avoiding permutation, and consider first its greedy staircase gridding. If this 128 gridding were to fail (G1) for some i ≥ 1, then we see from the two leftmost pictures in Figure 4 129 that the first entry of the (i + 2) nd cell could (and therefore, in a greedy gridding, would) have been 130 placed instead in the i th cell, a contradiction. On the other hand, if the gridding were to satisfy 131 (G1) but fail (G2) for some i ≥ 1 then we see from the two rightmost pictures in Figure 4 that the 132 first entry of the (i + 1) st cell would have been placed in the i th cell, another contradiction. 133 Next consider a staircase gridding π of π that satisfies (G1) and (G2). The condition (G2) implies 134 that the labels of the non-empty cells form an initial segment of P so we proceed inductively. By 135 definition, the entries of the 1 st cell form an initial increasing segment of π so we need to show that 136 it is the longest such segment. The next entry of π (reading left to right) must lie in the 2 nd cell 137 because (G1) shows that the leftmost entry of the 2 nd cell lies to the left of all entries of the 3 rd cell. 157 From any staircase gridding of a 321-avoiding permutation π we can construct a drawing of π on 158 these two parallel rays. First we add vertical and horizontal lines x = i and y = i for all natural 159 numbers i, splitting the figure into cells. To draw π on this figure, take any staircase gridding of 160 π and embed it cell by cell into the corresponding cells of the figure, making sure that the relative 161 order between entries in adjacent cells is preserved. An example is shown in Figure 5 . This is a regular language owing to Propositions 7.4 and 8.2 and the closure of the family of regular 796 languages under Boolean operations. Therefore C is in one-to-one correspondence with a regular 797 language. Moreover, if π ∈ C has length n then its image under the correspondence contains n 798 non-punctuation symbols. The generating function of a regular language over commuting variables 799 corresponding to its letters is a rational function and we can obtain the generating function for C 800 from that for G η c,C by replacing the variable corresponding to the punctuation symbol # by 1, and 801 those variables corresponding to non-punctuation symbols by x, so the generating function of C is 802 rational. 803 9. Well-Quasi-Ordered Subclasses 804 It remains to prove the second half of Theorem 1.1, namely that every well-quasi-ordered subclass 805 of 321-avoiding permutations has a rational generating function. This proof breaks naturally into 806 two parts. First we identify a necessary and sufficient condition for a subclass of Av(321) to be 807 well-quasi-ordered. Then we show, using arguments similar to those in the preceding section, that 808 this condition implies regularity of the corresponding languages. For the first part we identify a 809 particular antichain U ⊆ Av(321). Obviously, for a class C ⊆ Av(321), C ∩ U must be finite. It 810 happens that this condition is also sufficient. We begin with some preparatory remarks. 811 A permutation π is said to be sum decomposable if it can be written as a concatenation αβ where 812 every entry in the prefix α is smaller than every entry in the suffix β. If π has no non-trivial partition 813 of this form then it is said to be sum indecomposable. We may in this way interpret an arbitrary 814 permutation as a word over its sum indecomposable components (sum components for short). 815 Moving to a more general context, given a poset (P, ≤), the generalised subword order on P * is 816 defined by v ≤ w if there are indices 1 ≤ i 1 < i 2 < · · · < i |v| ≤ |w| such that v(j) ≤ w(i j ) for all j. 817 The following well-known result connects the well-quasi-ordering of P and P * . 818 Higman's Lemma  . If (P, ≤) is well-quasi-ordered then P * , ordered by the subword order, is 819 also well-quasi-ordered. 820 Returning to the context of permutations (and the containment order defined on them), Higman's 821 Lemma easily implies the following result. (For more details we refer the reader to Atkinson, Murphy, 822 and Ruškuc [11, Theorem 2.5].) 823 Proposition 9.1. Let C be a permutation class. If the sum indecomposable members of C are 824 well-quasi-ordered, then C is well-quasi-ordered.