Permutation patterns in genome rearrangement problems: the reversal model

TL;DR

This paper explores the relationship between permutation patterns and genome rearrangement models, specifically reversal and prefix reversal, providing characterizations and basis computations for permutations close to the identity.

Contribution

It introduces a novel characterization of permutations within a certain distance from the identity using generating peg permutations in the context of genome rearrangement models.

Findings

Characterization of permutations with distance ≤ k from the identity

Description of properties of the basis of permutation classes

Method to compute the basis for small values of k

Abstract

In the context of the genome rearrangement problem, we analyze two well known models, namely the reversal and the prefix reversal models, by exploiting the connection with the notion of permutation pattern. More specifically, for any $k$, we provide a characterization of the set of permutations having distance $\leq k$ from the identity (which is known to be a permutation class) in terms of what we call generating peg permutations and we describe some properties of its basis, which allow to compute such a basis for small values of $k$.

Equations

$$\{\begin{array}{l}{\Theta }_e(n)=n^{\bullet }(n-2{)}^{\bullet }\dots 4^{\bullet }{2}^{\bullet }{1}^{+}{3}^{\bullet }\dots (n-3{)}^{\bullet }(n-1{)}^{\bullet },\\ {\Lambda }_e(n)=(t+1{)}^{+}{t}^{\bullet }(t+2{)}^{\bullet }(t-1{)}^{\bullet }\dots (n-1{)}^{\bullet }{2}^{\bullet }{n}^{\bullet }{1}^{\bullet },\end{array}$$

$$\{\begin{array}{l}{\Theta }_o(n)=n^{\bullet }(n-2{)}^{\bullet }\dots 3^{\bullet }{1}^{-}{2}^{\bullet }{4}^{\bullet }\dots (n-3{)}^{\bullet }(n-1{)}^{\bullet },\\ {\Lambda }_o(n)=t^{-}(t+1{)}^{\bullet }(t-1{)}^{\bullet }(t+2{)}^{\bullet }\dots (n-1{)}^{\bullet }{2}^{\bullet }{n}^{\bullet }{1}^{\bullet },\end{array}$$

$$\begin{array}{rl}{\Theta }_e(n)=& \underset{P{R}_1}{\underbrace{n^{\bullet }(n-2{)}^{\bullet }\dots 4^{\bullet }{2}^{\bullet }{1}^{+}{3}^{\bullet }\dots (n-3{)}^{\bullet }(n-1{)}^{\bullet }}}\\ ⇝& \underset{P{R}_2}{\underbrace{(n-1{)}^{\bullet }(n-3{)}^{\bullet }\dots 3^{\bullet }{1}^{-}{2}^{\bullet }{4}^{\bullet }\dots (n-2{)}^{\bullet }}}{n}^{\bullet }\\ ⇝& (n-2{)}^{\bullet }\dots 4^{\bullet }{2}^{\bullet }{1}^{+}{3}^{\bullet }\dots (n-3{)}^{\bullet }(n-1{)}^{\bullet }{n}^{\bullet }=\\ & =[{\Theta }_e(n-2)](n-1{)}^{\bullet }{n}^{\bullet }:=\hat{\Theta }.\end{array}$$

$$prd({\Theta }_e(n))\le prd(\hat{\Theta })+2\le prd({\Theta }_e(n-2))+2=n-2+2=n,$$

$$\begin{array}{r}{\Lambda }_e(n)=\underset{PR}{\underbrace{(t+1{)}^{+}}}{t}^{\bullet }(t+2{)}^{\bullet }(t-1{)}^{\bullet }\dots (n-1{)}^{\bullet }{2}^{\bullet }{n}^{\bullet }{1}^{\bullet }\\ ⇝(t+1{)}^{-}{t}^{\bullet }(t+2{)}^{\bullet }(t-1{)}^{\bullet }\dots (n-1{)}^{\bullet }{2}^{\bullet }{n}^{\bullet }{1}^{\bullet }=\hat{\Lambda };\end{array}$$

$$prd({\Lambda }_e(n))\le prd(\hat{\Lambda })+1=prd({\Lambda }_o(n-1))+1=(n-1)+1=n.$$

$${\mathcal{M}}_{{\pi }^{\epsilon }}=\left\{\gamma :\gamma \text{mbox}isminimalsuchthatpeg(\gamma )={\pi }^{\epsilon }\text{mbox}andprd(\gamma )=prd({\pi }^{\epsilon })\right\}.$$

Full text

1 Introduction

One of the major trends in bioinformatics and biomathematics is the study of the genome rearrangement problem. Roughly speaking, given a genome, one is interested in understanding how the genome can evolve into another genome. To give a proper formalization, several models for rearranging a genome have been introduced, each of which defines a series of allowed elementary operations to be performed on a genome in order to obtain an adjacent one. For several models, it is possible to define a distance between two genomes, by counting the minimum number of elementary operations needed to transform one genome into the other. The investigation of the main properties of such a distance becomes then a key point in understanding the main features of the model under consideration.

A common formalization of any such models consists of encoding a genome using a permutation (in linear notation) and describing an elementary operation as a combinatorial operation on the entries of such a permutation. Many genome rearrangement models have been studied under this general framework. Among them, the following ones are very well known.

• •

The reversal model consists of a single operation, defined as follows: a new permutation is obtained from a given one by selecting a cluster of consecutive elements and reversing it. More formally, given $\pi=\pi_{1}\pi_{2}\cdots\pi_{n}$, a reversal is performed by choosing $i<j<n$ and then forming the permutation $\sigma=\pi_{1}\cdots\pi_{i-1}\boxed{\pi_{j}\pi_{j-1}\cdots\pi_{i+1}\pi_{i}}\pi_{j+1}\cdots\pi_{n}$. This model was introduced in [WEHM82], then studied for instance in [BP93, HP99].

• •

A variant of the reversal model is the prefix reversal model, which is a specialization of the previous one in which the reversal operation can only be performed on a prefix of the given permutation. This is also known as pancake sorting (see for instance [GP79]).

• •

A very popular and studied model is the transposition model, see [BP98]. Given a permutatation $\pi=\pi_{1}\cdots\pi_{n}$, a transposition operation consists of taking two adjacent clusters of consecutive elements and interchanging their positions. Formally, one has to choose indices $1\leq i<j<k\leq n+1$, then form the permutation $\sigma=\pi_{1}\cdots\pi_{i-1}\boxed{\pi_{j}\pi_{j+1}\cdots\pi_{k-1}}\boxed{\pi_{i}\pi_{i+1}\cdots\pi_{j-1}}\pi_{k}\cdots\pi_{n}$.

• •

As for the reversal, also for the transposition model there is a “prefix variant”. In the prefix transposition model the leftmost block of elements to interchange is a prefix of the permutation. Sorting by prefix transposition is studied in [DM02].

Independently from the chosen model, there are some general questions that can be asked in order to gain a better understanding of its combinatorial properties. First of all, the operations of the model often (but not always) allow to define a distance $d$ between two permutations $\rho$ and $\sigma$, as the minimum number of elementary operations needed to transform $\rho$ into $\sigma$. Moreover, when the operations are nice enough, the above distance $d$ could even be left-invariant, meaning that, given permutations $\pi,\rho,\sigma$ (of the same length), $d(\pi,\rho)=d(\sigma\pi,\sigma\rho)$. As a consequence, choosing for instance $\sigma=\rho^{-1}$, the problem of evaluating the distance $d(\pi,\rho)$ reduces to that of sorting $\pi$ with the minimum number of elementary allowed operations. Now, if $d$ is a left-invariant distance on the set $S_{n}$ of all permutations of the same length, define the $k$-ball of $S_{n}$ to be the set $B_{k}^{(d)}(n)=\{\rho\in S_{n}\;|\;d(\rho,id_{n})\leq k\}$, where $id_{n}$ is the identity permutation of length $n$. The following questions are quite natural to ask:

• •

compute the diameter of $B_{k}^{(d)}(n)$, i.e. the maximum distance between two permutations of $B_{k}^{(d)}(n)$;

• •

compute the diameter of $S_{n}$, i.e. the maximum distance between two permutations of $S_{n}$;

• •

characterize the permutations of $\partial B_{k}^{(d)}(n)$, i.e. the permutations of $B_{k}^{(d)}(n)$ having maximum distance from the identity;

• •

characterize the permutations of $\partial S_{n}$, i.e. the permutations of $S_{n}$ having maximum distance from the identity;

• •

characterize and enumerate the permutations of $B_{k}^{(d)}(n)$;

• •

design sorting algorithms and study the related complexity issues.

In the literature there are several results, concerning several evolution models, which give some insight into the above problems. Our work starts from the observation that, in many cases, the balls $B_{k}^{(d)}(n)$ can be characterized in terms of pattern avoidance. Recall that, given two permutations $\sigma\in S_{k}$ and $\tau=\tau_{1}\tau_{2}\cdots\tau_{n}\in S_{n}$, with $k\leq n$, we say that $\sigma$ is a pattern of $\tau$ when there exist $1\leq i_{1}<i_{2}<\cdots<i_{k}\leq n$ such that $\tau_{i_{1}}\tau_{i_{2}}\cdots\tau_{i_{k}}$ (as a permutation) is isomorphic to $\sigma$ (which means that $\tau_{i_{1}},\tau_{i_{2}},\ldots,\tau_{i_{k}}$ are in the same relative order as the elements of $\sigma$). This notion of pattern in permutations defines a partial order, and the resulting poset is known as the permutation pattern poset. When $\sigma$ is not a pattern of $\tau$, we say that $\tau$ avoids $\sigma$. A down-set $I$ (also called a permutation class) of the permutation pattern poset can be described in terms of its minimal excluded permutations (or, equivalently, the minimal elements of the complementary up-set): these permutations are called the basis of $I$. The idea of studying the balls $B_{k}^{(d)}(n)$ in terms of pattern avoidance is not new. As far as we know, the first model which has been investigated from this point of view is the (whole) tandem duplication-random loss model: Bouvel and Rossin [BR09] have in fact shown that, in such a model, the ball $B_{k}^{(d)}=\bigcup_{n\geq 0}B_{k}^{(d)}(n)$ is a class of pattern avoiding permutations, whose basis is the set of minimal permutations having $d$ descents (here minimal is intended in the permutation pattern order). Subsequent works [BP10, BF13, CGM11] have been done concerning the enumeration of the basis permutations of such classes. More recently, Homberger and Vatter [HV16] described an algorithm for the enumeration of any polynomial permutation class, which can be fruitfully used for all the above mentioned distances, since the resulting balls are indeed polynomial classes. However, their results do not allow to find information on the basis of the classes.

In the present work we try to enhance what have been obtained in [HV16] in two directions. First, we aim at giving a structural characterization of the balls for some of the above distances, thus complementing the results in [HV16], which is more concerned with computational issues. Secondly, we provide some insight on the properties of the bases of such balls, hoping to gain a better understanding of them. In particular, we will prove that the problem of determining what we call the clean compact peg basis of the balls (in both the reversal and the prefix reversal models) is decidable. The knowledge of the clean compact peg basis is of great help in finding the standard basis, as we will show in the next pages.

The companion paper [CF18], published in the proceedings of the conference GASCom 2018, deals with the cases of the block transposition and prefix block transposition models. In the present work we find analogous results for the reversal and prefix reversal model.

2 The reversal model

2.1 Generating permutations

We start with the study of the general reversal model. Recall (from the Introduction) that the reversal model is defined as the set of all permutations (of finite length) endowed with a combinatorial operation, called reversal, which consists of reversing a given segment (i.e. a set of adjacent elements in one-line notation) of a permutation, thus obtaining a new permutation of the same length. This operation defines a distance, called reversal distance, which is defined as the minimum number of reversals needed to transform one permutation into another. Since the reversal distance is left-invariant, we can limit ourselves to computing the distance of a given permutation from the identity permutation (having the correct length). Formally, such a quantity will be denoted $rd(\pi)$ (for a given permutation $\pi$), and the set of all permutations having distance $k$ from the identity (that is, the ball of radius $k$) will be denoted $B_{k}^{(rd)}$.

In order to deal with the reversal model, it will be useful to work both with standard permutations and with permutations whose elements are suitably decorated. Following [HV16], a peg permutation of length $n$ is a decorated permutation $\pi^{\varepsilon}=\pi_{1}^{\varepsilon_{1}}\pi_{2}^{\varepsilon_{2}}\cdots\pi_{n}^{\varepsilon_{n}}$, where each $\varepsilon_{i}$ is either $+,-$ or $\bullet$; for instance, $3^{+}4^{\bullet}1^{-}5^{-}2^{+}$ is a peg permutation of length 5. Given a peg permutation $\pi^{\varepsilon}=\pi_{1}^{\varepsilon_{1}}\pi_{2}^{\varepsilon_{2}}\cdots\pi_{n}^{\varepsilon_{n}}$, we denote with $\pi$ the underlying permutation $\pi=\pi_{1}\cdots\pi_{n}$. There is a particular subclass of peg permutations that will be useful in what follows. To define it, we need to introduce a few terminology.

An increasing strip of $\pi^{\varepsilon}$ is a maximal consecutive substring $\pi_{i}^{\varepsilon_{i}}\cdots\pi_{i+k-1}^{\varepsilon_{i+k-1}}$ such that, for all $j$, $\pi_{j+1}=\pi_{j}+1$ and $\varepsilon_{j}$ is either $+$ or $\bullet$. Similarly, a decreasing strip of $\pi^{\varepsilon}$ is a maximal consecutive substring $\pi_{i}^{\varepsilon_{i}}\cdots\pi_{i+k-1}^{\varepsilon_{i+k-1}}$ such that, for all $j$, $\pi_{j+1}=\pi_{j}-1$ and $\varepsilon_{j}$ is either $-$ or $\bullet$. When a sequence of consecutive entries is either an increasing or a decreasing strip, we will simply call it a strip. An identity peg permutation of length $n$ is an identity permutation in which every element is decorated with either $+$ or $\bullet$; notice that there are $2^{n}$ such permutations. Following again the terminology of [HV16], a clean compact peg permutation is a peg permutation all of whose strips have length 1. A weaker notion is that of a compact peg permutation, which is a peg permutation all of whose strips either have length 1 or are formed exclusively by elements decorated with $\bullet$. Thus, for instance, the above peg permutation $3^{+}4^{\bullet}1^{-}5^{-}2^{+}$ is not clean compact, whereas $3^{\bullet}4^{\bullet}1^{-}5^{-}2^{+}$ is compact, and $2^{+}5^{-}4^{+}1^{\bullet}3^{-}$ is a clean compact peg permutation of length 5. Any permutation $\pi$ can be associated with a unique clean compact peg permutation, denoted $peg(\pi)$, which is obtained by replacing each strip of $\pi$ with its minimum element decorated in a specific way, then suitably rescaling the resulting word. The decorations are the following:

if the element is the minimum of an increasing strip of length $\geq 2$, we decorate it with $+$; 2. 2.

if the element is the minimum of a decreasing strip of length $\geq 2$, we decorate it with $-$; 3. 3.

if the element is the unique element of a strip of length $1$, we decorate it with $\bullet$.

For instance, if $\pi=32451678$, then $peg(\pi)=2^{-}3^{+}1^{\bullet}4^{+}$.

We can extend the notion of reversal distance and pattern involvement to peg permutations. First of all, if we want to apply a reversal to a peg permutation $\pi^{\varepsilon}$, we perform the standard reversal to the underlying permutation $\pi$, then we invert the decoration $+$ and $-$ of each element involved in the operation (the $\bullet$ decorations remain unchanged). A similar notion of oriented reversal has already been considered in some classical works (see [HP99]). For example, if we apply the reversal of indices $2,4$ to the peg permutation $3^{+}1^{+}2^{-}5^{\bullet}4^{+}$, we obtain $3^{+}\underbrace{1^{+}2^{-}5^{\bullet}}_{rev}4^{+}\ =\ 3^{+}5^{\bullet}2^{+}1^{-}4^{+}$. In analogy with the usual definition, we will say that the reversal distance of a peg permutation $\pi^{\varepsilon}$ of length $n$ (from the identity) is the minimum number of (oriented) reversals needed to transform $\pi^{\varepsilon}$ in the corresponding identity permutation of length $n$. From now on, we will use the same term “reversal” for both standard and peg permutations, omitting the word “oriented” in the peg case. Moreover, we will use the same symbol $rd$ to denote the reversal distance for of peg permutations. The set of the peg permutations having distance at most $k$ from an identity permutation will be denoted with $\hat{B}_{k}^{(rd)}$.

Concerning the notion of pattern containment, given two peg permutations $\sigma^{\varepsilon}=\sigma_{1}^{\varepsilon_{1}}\cdots\sigma_{k}^{\varepsilon_{k}},\tau^{\varepsilon}=\tau_{1}^{\varepsilon_{1}}\cdots\tau_{n}^{\varepsilon_{n}}$, with $k\leq n$, we will say that $\sigma^{\varepsilon}$ is a pattern of $\tau^{\varepsilon}$ when there exist $1\leq i_{1}<i_{2}<\cdots<i_{k}\leq n$ such that:

$\tau_{i_{1}}\tau_{i_{2}}\cdots\tau_{i_{k}}$ (as a permutation) is isomorphic to $\sigma$; 2. 2.

if $\tau_{i_{j}}$ is decorated $+$ (resp. $-$), then $\sigma_{j}$ is decorated $+$ (resp. $-$).

Note that, for example, $1^{+}2^{\bullet}3^{+}$ is a pattern of $1^{+}2^{-}3^{+}$, while the converse is not true. The above relation is a partial order on the set of all peg permutations, and the resulting poset is called the peg permutation pattern poset. We have the following fundamental result, whose easy proof is left to the reader.

Proposition 2.1

If $\sigma$ is a pattern of $\tau$ and $\tau\in B_{k}^{(rd)}$, then $\sigma\in B_{k}^{(rd)}$; in other words, $B_{k}^{(rd)}$ is a down-set of the permutation pattern poset. 2. 2.

If $\sigma^{\varepsilon_{s}}$ is a pattern of $\tau^{\varepsilon_{t}}$ and $\tau^{\varepsilon_{t}}\in\hat{B}_{k}^{(rd)}$, then $\sigma^{\varepsilon_{s}}\in\hat{B}_{k}^{(rd)}$; in other words, $\hat{B}_{k}^{(rd)}$ is a down-set of the peg permutation pattern poset.

Now, given a peg permutation $\pi^{\varepsilon}=\pi_{1}^{\varepsilon_{1}}\pi_{2}^{\varepsilon_{2}}\cdots\pi_{n}^{\varepsilon_{n}}$, we can create a set of associated permutations by monotone inflations. A vector $v=(v_{1},\ldots,v_{n})$ of nonnegative integers is a legal inflation vector for $\pi^{\varepsilon}$ whenever $v_{i}\in\{0,1\}$ for each $i$ such that $\varepsilon_{i}=\bullet$; then, for a peg permutation $\pi^{\varepsilon}$ and a legal inflation vector $v$, we define the monotone inflation of $\pi^{\varepsilon}$ through $v$ as the permutation $\pi^{\varepsilon}[v]$ obtained from $\pi^{\varepsilon}$ in the following way:

if an element $\pi_{i}^{\varepsilon_{i}}$ is decorated $+$, we replace it with the identity permutation of length $v_{i}$; 2. 2.

if an element $\pi_{j}^{\varepsilon_{j}}$ is decorated $-$, we replace it with the reverse identity permutation of length $v_{j}$; 3. 3.

if an element $\pi_{k}^{\varepsilon_{k}}$ is decorated $\bullet$, we delete the element if $v_{k}=0$, otherwise we simply delete the sign $\varepsilon_{k}$; 4. 4.

finally, we suitably rescale each new insertion so to mantain the relative order of the elements of the permutation underlying $\pi^{\varepsilon}$.

If we perform the same construction, but we inflate using identity and reverse identity peg permutations, we obtain the peg monotone inflation of $\pi^{\varepsilon}$ through $v$, denoted $\pi^{\varepsilon}[v]_{peg}$; note that, in this case, we obtain a set of peg permutations for each choice of $v$. For instance, if $\pi^{\varepsilon}=3^{+}1^{+}2^{\bullet}5^{-}4^{\bullet}$ and $v=(2,0,1,3,1)$ (note that $v$ is a legal inflation vector for $\pi^{\varepsilon}$), then we have that:

$\pi^{\varepsilon}[v]=\underbrace{23}_{3^{+}}\underbrace{\ldots}_{1^{+}}\underbrace{1}_{2^{\bullet}}\underbrace{765}_{5^{-}}\underbrace{4}_{4^{\bullet}}$. 2. 2.

$\pi^{\varepsilon}[v]_{peg}\ni\underbrace{2^{\bullet}3^{+}}_{3^{+}}\underbrace{\ldots}_{1^{+}}\underbrace{1^{\bullet}}_{2^{\bullet}}\underbrace{7^{-}6^{\bullet}5^{-}}_{5^{-}}\underbrace{4^{\bullet}}_{4^{\bullet}}$.

From now on, again following [HV16], we will denote with $Grid(\pi^{\varepsilon})$ the grid class of $\pi^{\varepsilon}$, i.e. the set of all the monotone inflations of $\pi^{\varepsilon}$, and with $Grid(C)$ the set $\bigcup_{\pi^{\varepsilon}\in C}Grid(\pi^{\varepsilon})$, for a given set $C$ of peg permutations. The analogous classes for peg permutations will be denoted $Grid_{peg}(\pi^{\varepsilon})$ and $Grid_{peg}(C)$. The notion of grid class is compatible with the pattern involvement relation, in the sense specified by the following proposition. We give only a sketch of the proof, leaving the details to the reader.

Proposition 2.2

If $\pi^{\varepsilon}$ is a pattern of $\gamma^{\varepsilon^{\prime}}$, then $Grid(\pi^{\varepsilon})\subseteq Grid(\gamma^{\varepsilon^{\prime}})$ and $Grid_{peg}(\pi^{\varepsilon})\subseteq Grid_{peg}(\gamma^{\varepsilon^{\prime}})$. 2. 2.

If $\pi^{\varepsilon}\in\hat{B}_{k}^{(rd)}$, then $Grid(\pi^{\varepsilon})\subseteq B^{(rd)}_{k}$ and $Grid_{peg}(\pi^{\varepsilon})\subseteq\hat{B}_{k}^{(rd)}$.

Proof.  Given an occurrence of the pattern $\pi^{\varepsilon}$ in $\gamma^{\varepsilon^{\prime}}$, every monotone inflation of $\pi^{\varepsilon}$ can be obtained by inflating the elements of $\gamma^{\varepsilon}$ that form that occurrence of $\pi^{\varepsilon}$ exactly in the same way and the remaining ones with [math], so the first statement follows. To prove the second statement, observe that, given a sorting sequence of reversals for $\pi^{\varepsilon}$, one can find a new sorting sequence of the same length for every permutation $\gamma$ in $Grid(\pi^{\varepsilon})$: for each reversal $R$ of the given sorting sequence, define a reversal $R^{\prime}$ for $\gamma$ by incorporating in it all elements of $\gamma$ obtained by inflating each element of $\pi^{\varepsilon}$ affected by $R$. An analogous argument works also in the case of grid peg classes. $\blacksquare$

Corollary 2.3

Let $\pi=\pi_{1}\cdots\pi_{n}$ and $peg(\pi)=\alpha_{1}^{a_{1}}\cdots\alpha_{k}^{a_{k}}$ the associated clean compact peg permutation. Then $rd(\pi)\leq rd(peg(\pi))$.

Proof.  By construction, we have that $\pi\in Grid(peg(\pi))$, so $rd(\pi)\leq rd(peg(\pi))$ from the second statement of the above proposition. $\blacksquare$

Note that the opposite inequality is not true in general. For example, $rd(3412)=2$ (the sequence of reversals is $\underbrace{341}2\rightsquigarrow 1\underbrace{432}\rightsquigarrow 1234$), however the associated clean compact peg permutation is $2^{+}1^{+}$, that cannot be sorted with $2$ reversals.

The notions of monotone inflation and peg monotone inflation allow us to give a nice and precise description of $B_{k}^{(rd)}=\bigcup_{n\geq 0}B_{k}^{(rd)}(n)$. Our goal is to prove that $B_{k}^{(rd)}$ is the union of the grid classes of the clean compact permutations of $B_{k}^{(rd)}$ that are maximal with respect to the peg pattern involvement relation. To do this, we need some additional results and constructions.

Lemma 2.4

Let $\pi^{\varepsilon}$ be a clean compact peg permutation in $\hat{B}_{k}^{(rd)}$ that is maximal with respect to the peg pattern involvement relation; then $\pi^{\varepsilon}$ has no $\bullet$ decoration.

Proof.  Suppose there is an index $j$ such that $\pi_{j}$ is decorated $\bullet$ in $\pi^{\varepsilon}$. Given a sequence of (at most) $k$ reversals that sorts $\pi^{\varepsilon}$, let $t$ be the number of reversals that involve the element $\pi^{\bullet}_{j}$. If $t$ is even, then the same sequence of reversals also sorts the permutation $\bar{\pi}^{\varepsilon}$, obtained by decorating $\pi_{j}$ with $+$ instead of $\bullet$. This means that $\bar{\pi}^{\varepsilon}$ has distance (at most) $k$ and contains $\pi^{\varepsilon}$ as a pattern; moreover, $\bar{\pi}^{\varepsilon}$ is clean compact, because $\pi^{\varepsilon}$ is clean compact and we are replacing a $\bullet$ with a $+$. Thus we obtain a contradiction since $\pi^{\varepsilon}$ is supposed to be maximal. If $t$ is odd, we can argue in the same way by replacing the $\bullet$ with a $-$ decoration. $\blacksquare$

Suppose $\pi^{\varepsilon}=\pi_{1}^{\varepsilon_{1}}\cdots\pi_{n}^{\varepsilon_{n}}$ is a peg permutation of length $n$, with no elements decorated $\bullet$. Inflate $\pi^{\varepsilon}$ by choosing two (not necessarily distinct) indices $1\leq i\leq j\leq n$ and replacing $\pi_{i}^{\varepsilon_{i}}$ and $\pi_{j}^{\varepsilon_{j}}$ as follows:

if $i\neq j$, then replace $\pi_{i}$ with $\pi_{i}^{+}(\pi_{i}+1)^{+}$, if $\varepsilon_{i}=+$, or with $(\pi_{i}+1)^{-}\pi_{i}^{-}$ if $\varepsilon_{i}=-$; do the same for $\pi_{j}$; 2. 2.

if $i=j$, then replace $\pi_{i}$ with $\pi_{i}^{+}(\pi_{i}+1)^{+}(\pi_{i}+2)^{+}$, if $\varepsilon_{i}=+$; otherwise, if $\varepsilon_{i}=-$, replace it with $(\pi_{i}+2)^{-}(\pi_{i}+1)^{-}\pi_{i}^{-}$.

Now rescale the resulting string according to the relative order of the elements of $\pi$, so to obtain a new peg permutation of length $n+2$. If $I=\{i,j\}$, $i\leq j$, is the multiset of the selected indices, the resulting peg permutation will be denoted $\pi^{\varepsilon}_{I}$. A simple case by case analysis shows that, if the starting permutation $\pi^{\varepsilon}$ is clean compact, there exists a unique reversal $\rho_{I}$ for $\pi^{\varepsilon}_{I}$ such that the resulting permutation is clean compact as well; more specifically, $\rho_{I}$ is the reversal with indices $i+1,j+1$. Call $\tilde{\pi}^{\varepsilon}_{I}$ the permutation obtained by applying $\rho_{I}$ to $\pi^{\varepsilon}_{I}$. As an example, consider the peg permutation $\pi^{\varepsilon}=1^{+}2^{-}3^{+}$ and the multiset of indices $I=\{1,3\}$; then we get $\pi^{\varepsilon}_{I}=1^{+}2^{+}\ 3^{-}\ 4^{+}5^{+}$ and $\tilde{\pi}^{\varepsilon}_{I}=1^{+}\mathbf{4^{-}3^{+}2^{-}}5^{+}$. Choosing instead $J=\{2,2\}$, we get $\pi^{\varepsilon}_{J}=1^{+}\ 4^{-}3^{-}2^{-}\ 5^{+}$ and $\tilde{\pi}^{\varepsilon}_{J}=1^{+}4^{-}\mathbf{3^{+}}2^{-}5^{+}$. Note that in the two above cases we get the same resulting permutation; this shows how, in general, different choices of multisets can lead to the same permutation.

The following lemma sums up a few basic properties of the above construction that will be useful in the sequel. We leave the easy proof to the reader.

Lemma 2.5

Let $\pi^{\varepsilon}=\pi_{1}^{\varepsilon_{1}}\cdots\pi_{n}^{\varepsilon_{n}}$ be a clean compact peg permutation of length $n$ without $\bullet$, and $I$ a multiset of indices of $\pi$ of cardinality $2$. Then $\tilde{\pi}^{\varepsilon}_{I}$ is a clean compact peg permutation of length $n+2$ without $\bullet$. Moreover, if $\pi^{\varepsilon}_{1}=1^{+}$ and $\pi^{\varepsilon}_{n}=n^{+}$, then $(\tilde{\pi}^{\varepsilon}_{I})_{1}=1^{+}$ and $(\tilde{\pi}^{\varepsilon}_{I})_{n+2}=(n+2)^{+}$.

We are now ready to state the proposition that provides an inductive description of $B_{k}^{(rd)}$. The (excessively technical) proof is omitted.

Proposition 2.6

Let $\mathcal{I}(n)$ be the set of all multisets of cardinality $2$ of $\{1,2,\ldots n\}$ and let $\pi^{\varepsilon}$ be a clean compact peg permutation of length $n$ without $\bullet$. Denote with $Grid(\pi^{\varepsilon})^{+1}$ the set of all permutations that can be obtained by applying a single reversal to each permutation of $Grid(\pi^{\varepsilon})$ and with $Grid_{peg}(\pi^{\varepsilon})^{+1}$ the corresponding set of peg permutations. Then:

$Grid(\pi^{\varepsilon})^{+1}=\bigcup_{I\in\mathcal{I}(n)}Grid(\tilde{\pi}^{\varepsilon}_{I})$; 2. 2.

$Grid_{peg}(\pi^{\varepsilon})^{+1}=\bigcup_{I\in\mathcal{I}(n)}Grid_{peg}(\tilde{\pi}^{\varepsilon}_{I})$.

The above proposition tells that each permutation obtained by applying a single reversal to a permutation in $Grid(\pi^{\varepsilon})$ is a monotone inflation of $\tilde{\pi}^{\varepsilon}_{I}$, for some multiset $I$. An analogous fact holds in the poset of peg permutations.

A repeated applications of the above proposition leads to the following result.

Theorem 2.7

For every $k\geq 1$, there exist $N=N(k)$ clean compact peg permutations $\alpha^{\varepsilon^{1}}_{(1)},\ldots,\alpha^{\varepsilon^{N}}_{(N)}$, of length $2k+1$ and having distance $k$ from the identity, such that:

$B_{k}^{(rd)}=\bigcup_{j=1}^{N}Grid(\alpha^{\varepsilon^{j}}_{(j)})$; 2. 2.

$\hat{B}_{k}^{(rd)}=\bigcup_{j=1}^{N}Grid_{peg}(\alpha^{\varepsilon^{j}}_{(j)})$.

Such permutations will be called k-generating permutations and the set $\{\alpha_{(j)}^{\varepsilon_{j}}\,|\,j=1,\ldots,N\}$ is the $k$-generating set of $B_{k}^{(rd)}$ and $\hat{B}_{k}^{(rd)}$.

Proof.

We proceed by induction on $k$. Obviously $B_{0}^{(rd)}=Grid(1^{+})$. When $k=1$, it is easy to observe that $B_{1}^{(rd)}=Grid(1^{+}2^{-}3^{+})$, and $1^{+}2^{-}3^{+}$ is a clean compact peg permutation, has length $3$ and has distance $1$ from the identity. Now consider a permutation $\overline{\pi}\in B_{k+1}^{(rd)}\setminus B_{k}^{(rd)}$; this means, in particular, that there is a permutation $\pi\in B_{k}^{(rd)}$ such that $\overline{\pi}$ is obtained from $\pi$ by applying a single reversal. Thus, using the induction hypothesis, we can assert that there exists a clean compact peg permutation $\alpha^{\varepsilon}$, having length $2k+1$ and distance $k$ from the identity, such that $\pi\in Grid(\alpha^{\varepsilon})$. By Proposition 2.6, there exists $I\in\mathcal{I}(2k+1)$ such that $\overline{\pi}\in Grid(\tilde{\alpha}^{\varepsilon}_{I})$. Notice that, by Lemma 2.5, $\tilde{\alpha}^{\varepsilon}_{I}$ is a clean compact peg permutation of length $2k+3$.

It remains to prove that $\tilde{\alpha}^{\varepsilon}_{I}$ has distance $k+1$ from the identity. Clearly $rd(\tilde{\alpha}^{\varepsilon}_{I})\leq k+1$. Conversely, define a breakpoint of a peg permutation to be a pair of adjacent elements that is not part of a strip. In particular, analogously to what happens for standard permutations [HP99], a single reversal can change the number of breakpoints by at most $2$. Since $\tilde{\alpha}^{\varepsilon}_{I}$ is clean compact and has length $2k+3$, it has exactly $2k+2$ breakpoints, whereas an identity peg permutation of length $2k+3$ has no breakpoint. Therefore at least $\frac{2k+2}{2}=k+1$ reversals are needed to sort $\tilde{\alpha}^{\varepsilon}_{I}$. This means that $rd(\tilde{\alpha}^{\varepsilon}_{I})\geq k+1$, as desired. 2. 2.

The proof is identical to that of the previous case. Just notice that $\hat{B}_{0}^{(rd)}=\left\{1^{+},1^{\bullet}\right\}=Grid_{peg}(1^{+})$, so the base case of the induction argument is the same as before. $\blacksquare$

Notice that the maximum length of a clean compact peg permutation of $\hat{B}_{k}^{(rd)}$ is $2k+1$, since each reversal can create at most $2$ new breakpoints and an identity permutation has no breakpoint. Therefore, recalling that the permutations $\alpha^{\varepsilon^{1}}_{(1)},\ldots,\alpha^{\varepsilon^{N}}_{(N)}$ do not have elements decorated $\bullet$, we have that each generating permutation is maximal inside $\hat{B}_{k}^{(rd)}$ (because it has maximum length and no decoration $\bullet$) and clean compact (by construction). The next theorem proves the converse.

Theorem 2.8

Let $k\geq 1$ and let $\pi^{\varepsilon}$ be a maximal clean compact peg permutation in $\hat{B}_{k}^{(rd)}$. Then $\pi^{\varepsilon}$ is a $k$-generating permutation.

Proof.  The maximality of $\pi^{\varepsilon}$ guarantees that it has no element decorated $\bullet$; also notice that $\pi^{\varepsilon}$ has length $\leq 2k+1$, because it is clean compact and belongs to $\hat{B}_{k}^{(rd)}$. Moreover, Theorem 2.7 implies that there exists a generating permutation $\alpha^{\varepsilon_{a}}$, of length $2k+1$ and having distance $k$ from the identity, such that $\pi^{\varepsilon}\in Grid_{peg}(\alpha^{\varepsilon_{a}})$, i.e. there is a vector $v$ of nonnegative integers such that $\pi^{\varepsilon}\in\alpha^{\varepsilon_{a}}[v]_{peg}$. In particular, it must be $v_{i}\leq 1$ for each $i$, again because $\pi^{\varepsilon}$ is clean compact. As a consequence, we have that $\pi^{\varepsilon}$ is a pattern of $\alpha^{\varepsilon_{a}}$, but $\pi^{\varepsilon}$ is a maximal clean compact element of $\hat{B}_{k}^{(rd)}$, so it has to be $\pi^{\varepsilon}=\alpha^{\varepsilon_{a}}$, as desired. $\blacksquare$

Corollary 2.9

For every $k\geq 1$, $B_{k}^{(rd)}$ is the union of the grid classes of the maximal clean compact peg permutations of $\hat{B}_{k}^{(rd)}$. Thus $\alpha^{\varepsilon}$ is a $k$-generating permutations if and only if:

$rd(\alpha^{\varepsilon})=k$; 2. 2.

$\alpha^{\varepsilon}$* has length $2k+1$;* 3. 3.

$\alpha^{\varepsilon}$* is clean compact;* 4. 4.

$\alpha^{\varepsilon}$* has no decorations $\bullet$.*

The previous results suggest a procedure to list the generating permutations of $B_{k}^{(rd)}$: starting from $1^{+}$, perform $k$ successive monotone inflations as in Lemma 2.5 so to obtain all generating permutations of $B_{k}^{(rd)}$. This is similar to the approach used in [HV16]. For instance, when $k=2$, the generating set for $B_{2}^{(rd)}$ consists of the four permutations $1^{+}2^{-}3^{+}4^{-}5^{+},1^{+}4^{-}3^{+}2^{-}5^{+},1^{+}4^{+}2^{-}3^{-}5^{+},1^{+}3^{-}4^{-}2^{+}5^{+}$. As we have already observed, it is possible to obtain the same generating permutation several times. This is the main reason for which this approach cannot be used to enumerate the generating set.

Open problem 1. Enumerate the generating permutations of $B_{k}^{(rd)}$, for every $k$ (that is, determine the quantity $N(k)$ in the statement of Theorem 2.7).

2.2 The bases

As we have observed in Proposition 2.1, $B_{k}^{(rd)}$ and $\hat{B}_{k}^{(rd)}$ are down-sets in the permutation pattern poset and in the peg permutation pattern poset, respectively. Therefore it would be nice to find some information concerning their bases.

We consider the ball $\hat{B}_{k}^{(rd)}$ first and try to find some properties of its basis. For technical reasons that will become evident soon, in the sequel we restrict our attention to the poset of clean compact permutations as a subposet of the whole peg permutation pattern poset. Here we investigate properties of the set of minimal excluded permutations of $\hat{B}_{k}^{(rd)}$, which can be called the clean compact peg basis of $\hat{B}_{k}^{(rd)}$. Our goal is to determine an upper bound for the length of a basis permutation; such a bound can be used as a stopping condition for an algorithm which tries to explicitly describe the clean compact peg basis of $\hat{B}_{k}^{(rd)}$. This shows, in particular, that the problem of finding the clean compact peg basis is decidable. We start by proving some useful facts.

Lemma 2.10

Let $\pi^{\varepsilon}=\pi_{1}^{\varepsilon_{1}}\cdots\pi_{n}^{\varepsilon_{n}}$ be a clean compact peg permutation; then $\pi^{\varepsilon}$ contains at least one pattern $\gamma^{\varepsilon^{\prime}}$ of length $n-1$ which is clean compact as well.

Proof.  Consider the index $i$ such that $\pi_{i}=n$. If $i=1$ or $i=n$, then we can remove $\pi_{i}^{\varepsilon_{i}}$, thus obtaining a peg permutation that is still clean compact. Otherwise we need to consider a few distinct cases. Below we give details only for one of them, leaving the (analogous) remaining proofs to the reader. More specifically, we provide a complete analysis of the case in which $n$ is decorated $+$, and skip the cases in which $n$ is decorated $-$ or $\bullet$.

Suppose that $\pi_{i}^{\varepsilon_{i}}=n^{+}$. In most of the situations we can remove $n^{+}$ and obtain a clean compact peg permutations. There are only two “bad” cases, occurring when the two elements on the left and on the right of $n^{+}$ (respectively) form an increasing strip or a decreasing strip (of length 2). We now analyze only the first of the two above cases, the other one being completely analogous. Our hypothesis is therefore that $\pi^{\varepsilon}$ contains a string (of adjacent elements) of the form $k^{+/\bullet}\ n^{+}\ (k+1)^{+/\bullet}$, for some $k$. Here the decoration $+/\bullet$ stands for either a $+$ or a $\bullet$. In such a situation, removing $k+1$ almost always leads to a clean compact peg permutation, apart from two bad cases.

Suppose that $\pi^{\varepsilon}$ contains a string of the form $(k+2)^{-/\bullet}\ k^{\bullet}\ n^{+}\ (k+1)^{+/\bullet}$. Then we can remove $k$, unless we are in one of the two cases described below, which we will now manage separately.

• (i)

The first bad case occurs when $n=k+3$, and so $\pi^{\varepsilon}$ contains a string of the form $(k+2)^{\bullet}\ k^{\bullet}\ (k+3)^{+}\ (k+1)^{+/\bullet}$. We can then remove $k+2$, which always works except when there is a string of the form $(k-1)^{+/\bullet}\ (k+2)^{\bullet}\ k^{\bullet}\ (k+3)^{+}\ (k+1)^{+/\bullet}$. In such a case, however, we can always remove $k-1$ to obtain a clean compact peg permutation of length $n-1$.

• (ii)

The second bad case occurs when $\pi^{\varepsilon}$ contains a string of the form $(k+2)^{-/\bullet}\ k^{\bullet}\ n^{+}\ (k+1)^{\bullet}\ (k-1)^{-/\bullet}$. But now we can remove $k-1$ and we are done. 2. 2.

Suppose that $n=k+2$, and so $\pi^{\varepsilon}$ contains a string of the form $k^{+/\bullet}\ (k+2)^{+}\ (k+1)^{+/\bullet}$. So a good choice is to remove $k$, unless there is a string of the form $k^{+/\bullet}\ (k+2)^{+}\ (k+1)^{\bullet}\ (k-1)^{-/\bullet}$. In this case, however, one could remove $k-1$, unless there is a string of the form $(k-2)^{+/\bullet}\ k^{+/\bullet}\ (k+2)^{+}\ (k+1)^{\bullet}\ (k-1)^{-/\bullet}$. Now it turns out that this argument can be repeated, so that we can always find an element to remove, unless we have already considered all the elements of the permutation. If this happens, then $\pi^{\varepsilon}$ has $1$ either as its first or its last element, so $1$ can be removed. $\blacksquare$

Lemma 2.11

Let $\pi^{\varepsilon}=\pi_{1}^{\varepsilon_{1}}\cdots\pi_{n}^{\varepsilon_{n}}$ be a permutation in the clean compact peg basis of $\hat{B}_{k}^{(rd)}$. Then $\pi_{1}^{\varepsilon_{1}}\neq 1^{+/\bullet}$ and $\pi_{n}^{\varepsilon_{n}}\neq n^{+/\bullet}$.

Proof. If $\pi_{1}^{\varepsilon_{1}}=1^{+}$ or $\pi_{1}=1^{\bullet}$, then we could remove it without changing the reversal distance of $\pi^{\varepsilon}$ , against the minimality of $\pi^{\varepsilon}$. The case $\pi_{n}^{\varepsilon_{n}}=n^{+/\bullet}$ is analogous. $\blacksquare$

We can now state our main result concerning the length of a basis permutation.

Theorem 2.12

Every permutation belonging to the clean compact peg basis of $\hat{B}_{k}^{(rd)}$ has length at most $2k+1$.

Proof.  It is easy to prove that every basis permutation $\pi^{\varepsilon}$ has length at most $2k+2$. Indeed, suppose that $\pi^{\varepsilon}$ has length (at least) $2k+3$. By Lemma 2.10, $\pi^{\varepsilon}$ contains a clean compact pattern $\gamma^{\varepsilon^{\prime}}$ of length (at least) $2k+2$, that must belong to $\hat{B}_{k}^{(rd)}$ (thanks to the minimality property of $\pi^{\varepsilon}$). But the maximum length of a clean compact permutation in $\hat{B}_{k}^{(rd)}$ is $2k+1$, so we have a contradiction.

Now suppose that $\pi^{\varepsilon}$ has length $2k+2$; again, $\pi^{\varepsilon}$ contains a clean compact pattern $\gamma^{\varepsilon^{\prime}}$ of length $2k+1$, with $\gamma^{\varepsilon^{\prime}}\in\hat{B}_{k}^{(rd)}$.

By Theorem 2.7, there exists a $k$-generating permutation $\alpha^{a}$ such that $\gamma^{\varepsilon^{\prime}}\in\alpha^{a}[v]$, for some legal inflation vector $v$. In particular, $v_{i}\leq 1$ for each $i$, because $\gamma^{\varepsilon^{\prime}}$ is clean compact; moreover, both $\gamma^{\varepsilon^{\prime}}$ and $\alpha^{a}$ have length $2k+1$, so it must be $v_{i}=1$, for each $i$. As a consequence of Lemma 2.5, we have that $\gamma_{1}^{\varepsilon^{\prime}_{1}}=1^{+/\bullet}$ and $\gamma_{2k+1}^{\varepsilon^{\prime}_{2k+1}}=(2k+1)^{+/\bullet}$, whereas Lemma 2.11 implies that $\pi_{1}^{\varepsilon_{1}}\neq 1^{+/\bullet}$ and $\pi_{2k+2}\neq(2k+2)^{+/\bullet}$. Since $\gamma^{\varepsilon^{\prime}}$ is obtained from $\pi^{\varepsilon}$ by removing a single element, there are only two possible cases:

• (i)

$\pi^{\varepsilon}=(2k+2)^{\varepsilon_{1}}1^{+/\bullet}\pi_{3}^{\varepsilon_{3}}\cdots\pi_{2k+1}^{\varepsilon_{2k+1}}(2k+1)^{+/\bullet}$ and $\gamma^{\varepsilon^{\prime}}$ is obtained by removing the first element $2k+2$. Then the permutation $\beta^{b}$ obtained from $\pi^{\varepsilon}$ by removing $1^{+/\bullet}$ would be a clean compact peg permutation of length $2k+1$ of $\hat{B}_{k}^{(rd)}$. Therefore, again using Lemma 2.5, it should be $\beta_{1}^{b_{1}}=1^{+/\bullet}$, which gives a contradiction.

• (ii)

$\pi^{\varepsilon}=2^{+/\bullet}\pi_{2}^{\varepsilon_{2}}\cdots\pi_{2k}^{\varepsilon_{2k}}(2k+2)^{+/\bullet}1^{\varepsilon_{2k+2}}$ and $\gamma^{\varepsilon^{\prime}}$ is obtained by removing the last element $1$. This case is clearly symmetric to the previous one: now we can remove $2k+2$ and use the same arguments. $\blacksquare$

Corollary 2.13

The clean compact peg basis of $\hat{B}_{k}^{(rd)}$ is finite, for every $k\geq 1$.

The above theorem also suggests a procedure to determine the clean compact peg basis of $\hat{B}_{k}^{(rd)}$ (thus showing that such a problem is decidable). For each clean compact peg permutation $\pi^{\varepsilon}$ of length $2k+1$ which is not $k$-generating, take the set of permutations it covers:

if all of them lie below some $k$-generating permutation, then $\pi^{\varepsilon}$ is in the clean compact peg basis of $\hat{B}_{k}^{(rd)}$;

• -

otherwise, repeat the same procedure starting from the permutations covered by $\pi^{\varepsilon}$ which do not belong to $\hat{B}_{k}^{(rd)}$.

As an instance, we have the following result.

Proposition 2.14

The clean compact peg basis of $\hat{B}_{1}^{(rd)}$ is $\{1^{-}2^{-},2^{+}1^{\bullet},2^{\bullet}1^{+}\}$.

Proof. Since $\hat{B}_{1}^{(rd)}=Grid_{peg}(1^{+}2^{-}3^{+})$, we perform the above procedure with each clean compact peg permutations of length $3$, except for $1^{+}2^{-}3^{+}$. A direct inspection shows that there are no such permutations covering only elements of $\hat{B}_{1}^{(rd)}$. Moreover $1^{-}2^{-},2^{+}1^{\bullet},2^{\bullet}1^{+}$ are the only clean compact peg permutations of length $2$ which are not in $\hat{B}_{1}^{(rd)}$ and such that all of their coverings are in $\hat{B}_{1}^{(rd)}$. $\blacksquare$

Unfortunately, translating the above results into the whole peg permutation pattern poset seems to be very hard. The main problem is that there exist permutations in the peg basis which are not clean compact; for example, the permutation $2^{\bullet}1^{\bullet}4^{\bullet}3^{\bullet}$ has distance $2$ and is minimal with respect to this property: it is in fact a peg basis element of $\hat{B}_{1}^{(rd)}$. Also notice that both $2^{\bullet}1^{\bullet}4^{\bullet}3^{\bullet}$ and $1^{-}2^{-}$ are elements of the peg basis, but $2^{\bullet}1^{\bullet}4^{\bullet}3^{\bullet}\in Grid(1^{-}2^{-})$; this suggests that there are some intrinsic difficulties in giving a satisfactory description of the peg basis, which means that perhaps the whole peg permutation pattern poset is not the right setting for this problem. Moreover it seems to be quite hard to find a nontrivial upper bound for the length of the permutations of the peg basis, although we know that such a basis is still finite as a consequence of some general results concerning grid classes (see [AABRV13]). A useful characterization provided in [HV16] guarantees that a permutation of the peg basis is at least compact.

Lemma 2.15

[HV16]** Let $\pi^{\varepsilon}$ be a peg permutation; then $\pi^{\varepsilon}$ is compact if and only if $Grid(\pi^{\varepsilon})$ properly contains $Grid(\tau^{t})$, for each $\tau^{t}<\pi^{\varepsilon}$ in the peg permutation pattern poset.

Proposition 2.16

Let $\pi^{\varepsilon}$ be a peg basis permutation of $\hat{B}_{k}^{(rd)}$. Then $\pi^{\varepsilon}$ is compact.

Proof.  Suppose that $\pi^{\varepsilon}$ contains a proper peg pattern $\tau^{t}$ such that $Grid(\tau^{t})=Grid(\pi^{\varepsilon})$. Then Proposition 2.1 implies that $rd(\tau^{t})\leq rd(\pi^{\varepsilon})=k+1$. Moreover it cannot be $rd(\tau^{t})\leq k$, otherwise we would also have $rd(\pi^{\varepsilon})\leq k$, because $\pi^{\varepsilon}\in Grid(\tau^{t})$ by hypothesis. Thus $rd(\pi^{\varepsilon})=rd(\tau^{t})=k+1$ and $\tau^{t}<\pi^{\varepsilon}$, which gives a contradiction because $\pi^{\varepsilon}$ is minimal outside $\hat{B}_{k}^{(rd)}$. Therefore, thanks to the above lemma, we can conclude that $\pi^{\varepsilon}$ is compact. $\blacksquare$

The final part of the present section is devoted to the standard basis of $B_{k}^{(rd)}$. The general theory of geometric grid classes allows to say that $B_{k}^{(rd)}$ is a permutation class having finite basis; moreover $B_{k}^{(rd)}$ is strongly rational, meaning that its generating function is rational, together with the generating functions of all of its subclasses [AABRV13]. Here we sketch a description the basis of $B_{k}^{(rd)}$, starting from the knowledge of the clean compact peg basis of $\hat{B}_{k}^{(rd)}$.

Let $k\geq 1$ and suppose that $\{\beta_{1}^{b_{1}},\dots,\beta_{N}^{b_{N}}\}$ is the clean compact peg basis of $\hat{B}_{k}^{(rd)}$. For every $i=1,\dots,N$, define the following sets of (standard) permutations:

$\mathcal{A}_{\beta_{i}^{b_{i}}}=\left\{\pi:\ peg(\pi)=\beta_{i}^{b_{i}}\right\}$;

• -

$\mathcal{M}_{\beta_{i}^{b_{i}}}=\left\{\pi\in\mathcal{A}_{\beta_{i}^{b_{i}}}:\pi\mbox{ is minimal such that }rd(\pi)=rd(\beta_{i}^{b_{i}})\right\}$;

• -

$\mathcal{M}=\bigcup_{i}\mathcal{M}_{\beta_{i}^{b_{i}}}$.

Our goal is to show that $B_{k}^{(rd)}=Av(\mathcal{M})$, so that the minimal elements in $\mathcal{M}$ form the basis of $B_{k}^{(rd)}$. First of all, we prove that the sets $\mathcal{M}_{\beta_{i}^{b_{i}}}$ are not empty.

Lemma 2.17

Let $\pi^{\varepsilon}$ be a clean compact peg permutation such that $rd(\pi^{\varepsilon})=k$. Then there is a permutation $\gamma\in Grid(\pi^{\varepsilon})$ such that $rd(\gamma)=k$ and $peg(\gamma)=\pi^{\varepsilon}$.

Proof.  Let $\gamma=\pi^{\varepsilon}[v]$, with $v$ defined as follows:

$v_{i}=1$, for each $i$ such that $\pi_{i}$ is decorated $\bullet$;

• -

$v_{i}=N>1$, otherwise.

Note that $v$ is a legal inflation vector for $\pi^{\varepsilon}$. Suppose that $rd(\gamma)\leq k-1$. Theorem 2.7 implies that there exists a $(k-1)$-generating permutation $\alpha^{a}$ such that $\gamma\in Grid(\alpha^{a})$. In particular, $\alpha^{a}$ is a clean compact peg permutation of length $2k-1$. Depending on the types of decorations $\pi^{\varepsilon}$ contains, we can distinguish two cases. If $\pi^{\varepsilon}$ only contains $\bullet$, the underlying permutation $\pi$ is $\gamma$ itself and $rd(\pi)=rd(\pi^{\varepsilon})=k$, which is in contradiction with the assumption $rd(\gamma)\leq k-1$. If instead $\pi^{\varepsilon}$ contains at least one decoration different from $\bullet$, we observe that there is at least one element $\pi_{j}^{\varepsilon_{j}}$ of $\pi^{\varepsilon}$ such that the corresponding strip $S_{j}$ in $\gamma$ is not contained in the inflation of a single element of $\alpha^{a}$: otherwise, in fact, we could write $\pi^{\varepsilon}$ as an appropriate inflation of $\alpha^{a}$, so it would be $\pi^{\varepsilon}\in Grid_{peg}(\alpha^{a})$ and thus $rd(\pi)\leq k-1$, which is not. In particular, the decoration $\varepsilon_{j}$ is either $+$ or $-$, because the corresponding strip $S_{j}$ must contain at least $2$ elements. Moreover, if $S_{j}$ is increasing, its elements are necessarily distributed among the inflations of entries of $\alpha^{a}$ which are decorated either $-$ or $\bullet$, otherwise they could be assigned to the inflation of a single element decorated $+$ and so again $\pi^{\varepsilon}\in Grid_{peg}(\alpha^{a})$. An analogous argument holds if $S_{j}$ is decreasing. Hence each element of $S_{j}$ must belong to a different strip of $\alpha^{a}$. Now, recalling that $\alpha^{a}$ has length $|\alpha^{a}|=2k-1$, if we choose $N>2k-1$, we have that the $N$ elements of $S_{j}$ have to be distributed among the inflations of $N>|\alpha^{a}|$ different element of $\alpha^{a}$, which is impossible. $\blacksquare$

In other words, the above lemma says that the reversal distance of a peg permutation is equal to the maximum distance of a permutation in its standard grid class: every permutation in the grid class of $\pi^{\varepsilon}$ has distance at most $rd(\pi^{\varepsilon})$ (by Proposition 2.2) and we have proved that there is at least one permutation that attains this value. As an example, consider the peg permutation $2^{+}1^{+}$, which has distance $3$. The generating permutations of $\hat{B}_{2}^{(rd)}$ have length $5$, so we can choose $N=6>5$. Define $\gamma=2^{+}1^{+}\cdot[6,6]=\underbrace{789\ 10\ 11\ 12}_{S_{1}}\ \underbrace{123456}_{S_{2}}$. It cannot be $rd(\gamma)=2$, since the six elements of either $S_{1}$ or $S_{2}$ should belong to six distinct inflations of elements of a $2$-generating permutation, whose length is 5.

The next result is an immediate consequence of the above lemma.

Corollary 2.18

If $\beta^{b}$ belongs to the clean compact peg basis of some $\hat{B}_{k}^{(rd)}$, then the set $\mathcal{M}_{\beta^{b}}$ is not empty.

We are now able to prove that $B_{k}^{(rd)}=Av(\mathcal{M})$. First of all, every permutation in $\mathcal{M}$ has distance equal to some permutation of the clean compact peg basis of $\hat{B}_{k}^{(rd)}$; therefore, if $\pi\in\mathcal{M}$, then $\pi\notin B_{k}^{(rd)}$. Moreover, we have to show that each permutation $\gamma\notin B_{k}^{(rd)}$ contains some permutation $\pi\in\mathcal{M}$ as a pattern. If $\gamma\notin B_{k}^{(rd)}$, then also $peg(\gamma)\notin\hat{B}_{k}^{(rd)}$ by Corollary 2.3. Since $peg(\gamma)$ is clean compact, there is a clean compact peg basis permutation $\beta_{i}^{b_{i}}$, for some $i$, such that $peg(\gamma)\geq\beta_{i}^{b_{i}}$. Given an occurrence of $\beta_{i}^{b_{i}}$ in $peg(\gamma)$, consider the permutation $\hat{\gamma}$ obtained by taking only the strips of $\gamma$ corresponding to the elements of $\beta_{i}^{b_{i}}$. Clearly $\hat{\gamma}\leq\gamma$ and moreover $peg(\hat{\gamma})=\beta_{i}^{b_{i}}$ by construction, so $\hat{\gamma}\in\mathcal{A}_{\beta_{i}^{b_{i}}}$ and there exists a minimal permutation $\pi\in\mathcal{M}_{\beta_{i}^{b_{i}}}$ such that $\hat{\gamma}\geq\pi$. Thus we have $\gamma\geq\hat{\gamma}\geq\pi$, as desired.

Example. Consider the ball $\hat{B}_{1}^{(rd)}$, whose clean compact peg basis is $\{1^{-}2^{-},2^{+}1^{\bullet},2^{\bullet}1^{+}\}$. Here we have $\mathcal{M}_{1^{-}2^{-}}=\left\{2143\right\}$, $\mathcal{M}_{2^{+}1^{\bullet}}=\left\{231\right\}$ and $\mathcal{M}_{2^{\bullet}1^{+}}=\left\{312\right\}$, so $B_{1}^{(rd)}=Av(2143,231,312)$ and the basis is indeed $\{2143,231,312\}$. Notice that 2143 is the minimum of the set $\mathcal{A}_{1^{-}2^{-}}$, and an analogous fact holds for 231 and 312 as well. However, this is not true in general. For instance, $2^{+}1^{+}$ belongs to the clean compact peg basis of $\hat{B}_{2}^{(rd)}$, but the minimum permutation in $\mathcal{A}_{2^{+}1^{+}}$ is $3412$, which has distance $2$ (unlike $2^{+}1^{+}$, which has distance 3). So $3412\notin\mathcal{M}_{2^{+}1^{+}}$. Moreover (see Figure 1) the permutations covering 3412 in $\mathcal{A}_{2^{+}1^{+}}$ are $34512$ and $45123$, which have distance $2$ as well. Therefore, in order to find the minimal permutations of $\mathcal{A}_{2^{+}1^{+}}$ at distance $3$, we have to reach length $6$, where it can be shown that $456123$ is in fact minimal at distance $3$. Thus $456123$ is a basis element corresponding to the clean compact peg basis permutation $2^{+}1^{+}$.

One nice feature of this approach is that the sets $\mathcal{A}_{\beta_{i}^{b_{i}}}$ are disjoint. Thus each basis permutation is generated by precisely one of such sets. On the other hand, it is still not clear if a single set $\mathcal{M}_{i}$ can lead to multiple basis permutations.

Open problem 2. For a given clean compact peg basis permutation $\beta^{b}$, determine $|\mathcal{M}_{\beta^{b}}|$. Is there always a single minimal permutation?

Open problem 3. Is the set $\mathcal{M}$ an antichain of permutations? In other words, is $\mathcal{M}$ the standard basis?

Open problem 4. Determine a non trivial bound for the length of a basis permutation of $B_{k}^{(rd)}(k)$, for $k\geq 1$.

3 The prefix reversal model

3.1 Generating permutations

In this section we focus on prefix reversals, i.e. reversals that only affect the initial portion of a permutation. Denoting with $prd$ the prefix reversal distance, our first goal is to obtain a characterization of the balls $B_{k}^{(prd)}$ in terms of generating permutations, similarly to what we did in the case of the standard reversal distance. As a first example, it is easy to see that $B_{1}^{(prd)}=Grid(1^{-}2^{+})$, so that $1^{-}2^{+}$ is the only generating permutation. By specializing the recursive construction developed in the case of general reversals, we can explicitly determine all the $(k+1)$-generating permutations starting from the $k$-generating ones.

Proposition 3.1

Let $\pi^{\varepsilon}=\pi_{1}^{\varepsilon_{1}}\cdots\pi_{n}^{\varepsilon_{n}}$ be a generating permutation of $B_{k}^{(prd)}$ and let $i\in\{1,\dots,n\}$.

If $\pi^{\varepsilon}=\alpha\ \pi_{i}^{+}\ \beta$, for some peg permutations $\alpha,\beta$, then $\pi_{i}^{-}\dot{\alpha}^{R}(\pi_{i}+1)^{+}\dot{\beta}$ is a generating permutation of $B_{k+1}^{(prd)}$, where $\dot{\alpha},\dot{\beta}$ are obtained from $\alpha,\beta$ (respectively) by increasing by $1$ all the entries that are greater than $\pi_{i}$ and $\dot{\alpha}^{R}$ is the reversal of $\dot{\alpha}$. 2. 2.

If $\pi^{\varepsilon}=\alpha\ \pi_{i}^{-}\ \beta$, for some peg permutations $\alpha,\beta$, then $(\pi_{i}+1)^{+}\dot{\alpha}^{R}(\pi_{i})^{-}\dot{\beta}$ is a generating permutation of $B_{k+1}^{(prd)}$, where $\dot{\alpha},\dot{\beta}$ and $\dot{\alpha}^{R}$ are defined as above.

Moreover, every $(k+1)$-generating permutation is obtained from a $k$-generating permutation by performing one of the two above constructions.

Proof (sketch). Since the prefix reversal model is a special case of the reversal one, we have that every generating permutation for $B_{k+1}^{(prd)}$ can be obtained from a $k$-generating permutation $\pi^{\varepsilon}$ by choosing an index $i$, then suitably inflating $\pi_{i}^{\varepsilon_{i}}$, according to its decoration, and finally performing the prefix reversal operation that removes the new strip. This is done in analogy with the construction described before Lemma 2.5, with the only difference that the first index of the multiset has to be $1$ since we are considering prefix reversals. $\blacksquare$

The above proposition gives a recipe for constructing the generating permutations of $\hat{B}_{k+1}^{(prd)}$ starting from those of $\hat{B}_{k}^{(prd)}$. Notice that, if $\pi^{\varepsilon}$ has length $m$, then the permutations obtained with the previous construction have length $m+1$. Since $B_{1}^{(prd)}=Grid(1^{-}2^{+})$, a simple inductive argument shows that the generating permutations of $B_{k}^{(prd)}$ have length $k+1$. Actually, we have something more, which is the analogue of Corollary 2.9 in the case of the prefix reversal model. Since the proof is similar, we just state it.

Theorem 3.2

For every $k\geq 1$, the generating set of $B_{k}^{(prd)}$ is the set of all maximal clean compact peg permutations of $\hat{B}_{k}^{(prd)}$.

However, in the prefix case, we are able to count generating permutations.

Theorem 3.3

The generating set of $B_{k}^{(prd)}$ has cardinality $k!$.

Proof. Observe that, if $\pi^{\varepsilon}=\pi_{1}^{\varepsilon_{1}}\cdots\pi_{n}^{\varepsilon_{n}}$ is a $(k+1)$-generating permutation, then it has been obtained from a $k$-generating permutation by one of the constructions described in Proposition 3.1. However, $\pi^{\varepsilon}$ cannot be obtained in two different ways. This can be shown by considering the elements $\pi_{1}^{\varepsilon_{1}}$:

if $\pi_{1}^{\varepsilon_{1}}=a^{-}$, for some $a\geq 1$, then $\pi^{\varepsilon}$ is obtained as in $1.$ of Proposition 3.1, when $\pi_{i}^{\epsilon_{i}}=a^{+}$; 2. 2.

if $\pi_{1}^{\varepsilon_{1}}=a^{+}$, for some $a\geq 1$, then $\pi^{\varepsilon}$ is obtained as in $2.$ of Proposition 3.1, when $\pi_{i}^{\epsilon_{i}}=(a-1)^{-}$.

Since the above cases are disjoint, we can conclude that $\pi^{\varepsilon}$ comes from a unique generating permutation of $B_{k}^{(prd)}$ through the construction of Proposition 3.1. Thus, the total number of generating permutations of $B_{k+1}^{(prd)}$ is obtained by multiplying the number of generating permutations of $B_{k}^{(prd)}$ by the number of possible inflations of each of them, which is $k+1$. Since the generating set of $B_{1}^{(prd)}$ has cardinality $1$, a simple inductive argument shows that the required cardinality is indeed $\prod_{i=1}^{k}{i}=k!$. $\blacksquare$

We have already observed that, for $k=1$, the generating set is $\{1^{-}2^{+}\}$. For $k=2$, the generating set is $\{2^{+}1^{-}3^{+},2^{-}1^{+}3^{+}\}$, while for $k=3$ we obtain the set $\{2^{-}3^{+}1^{-}4^{+},2^{+}3^{-}1^{-}4^{+},3^{-}1^{+}2^{-}4^{+},3^{+}2^{-}1^{+}4^{+},1^{-}3^{+}2^{+}4^{+},3^{-}1^{-}2^{+}4^{+}\}$.

3.2 The bases

Concerning the clean compact peg basis of $\hat{B}_{k}^{(prd)}$, we are able to prove an analogue of Theorem 2.12, although we need to take care of some exceptions.

Lemma 3.4

Let $\pi^{\varepsilon}$ be a clean compact peg permutation of length $n$ that does not end with its maximum value. Then $prd(\pi^{\varepsilon})\geq n$.

Proof.  Since any prefix reversal can change the number of breakpoints of a peg permutation by at most $1$ and a clean compact peg permutation of length $n$ has exactly $n-1$ breakpoints, we have that $prd(\pi^{\varepsilon})\geq n-1$. Moreover, since $\pi^{\varepsilon}$ does not end with $n$, in any sequence of prefix reversals that transforms the identity in $\pi^{\varepsilon}$ there must be a prefix reversal $R$ that moves $n$ from the end. This means that $R$ reverses the entire permutation. In particular, $R$ does not modify the number of breakpoints, so we need at least $(n-1)+1=n$ reversals to obtain $\pi^{\varepsilon}$. $\blacksquare$

In the proof of the next theorem the software PermLab [AL] has been of great help.

Theorem 3.5

Let $k\geq 0$ and $n=k+2$. The following are clean compact peg basis permutations of $\hat{B}_{k}^{(prd)}$:

[TABLE]

if $n$ is even and $t=\frac{n}{2}$; 2. 2.

[TABLE]

if $n$ is odd and $t=\frac{n+1}{2}$.

Moreover, each of the above peg permutations has distance $n=k+2$.

In the sequel, these peg permutations will be called exceptional.

Proof. The proof will use induction on $k$. If $k=0$, we get $\Theta_{e}(2)=2^{\bullet}1^{+}$ and $\Lambda_{e}(2)=2^{+}1^{\bullet}$, which are minimal clean compact peg permutations in the complement of $\hat{B}_{0}^{(prd)}$ and have distance $2$. If $k=1$, we get $\Theta_{o}(3)=3^{\bullet}1^{-}2^{\bullet}$ and $\Lambda_{o}(3)=2^{-}3^{\bullet}1^{\bullet}$, again minimal clean compact peg permutations in the complement of $\hat{B}_{1}^{(prd)}$ and having distance $3$.

If $k\geq 2$, it is easy to observe that the above peg permutations are clean compact of length $n=k+2$ and do not end with their maximum value. Thus, by Lemma 3.4, they have distance at least $n=k+2$.

Next, we prove that the exceptional permutations are minimal in the complement of $\hat{B}_{k}^{(prd)}$. In other words, we prove that each clean compact pattern of one of these permutations has distance at most $k$. We first consider $\Theta_{e}(n)=n^{\bullet}(n-2)^{\bullet}\dots 4^{\bullet}2^{\bullet}1^{+}3^{\bullet}\dots(n-3)^{\bullet}(n-1)^{\bullet}$, with $n$ even. There are several ways to obtain a clean compact pattern.

If we remove $i^{\bullet}$, with $i\neq 1,n$, in order to obtain a clean compact peg permutation, we also need to remove either $i+1$ (if $i$ is even) or $i-1$ (if $i$ is odd); otherwise, in fact, we would obtain the strip $(i-1),i$ or $i,(i-1)$, respectively. We thus obtain the permutation $(n-2)^{\bullet}(n-4)^{\bullet}\dots 4^{\bullet}2^{\bullet}1^{+}3^{\bullet}\dots(n-5)^{\bullet}(n-3)^{\bullet}=\Theta_{e}(n-2)$, which has distance $n-2=k$ by induction hypothesis. 2. 2.

If we replace $1^{+}$ with $1^{\bullet}$, we obtain the pattern $n^{\bullet}(n-2)^{\bullet}\dots 4^{\bullet}2^{\bullet}1^{\bullet}3^{\bullet}\dots(n-3)^{\bullet}(n-1)^{\bullet}$, which is not clean compact because of the strip $2^{\bullet}1^{\bullet}$. In order to obtain a clean compact pattern, we have to remove either $2^{\bullet}$ or $1^{\bullet}$, but in both cases, after rescaling, we get the permutation $(n-1)^{\bullet}(n-3)^{\bullet}\dots 3^{\bullet}1^{\bullet}2^{\bullet}\dots(n-4)^{\bullet}(n-2)^{\bullet}$, which contains $1^{\bullet}2^{\bullet}$. Iterating this argument, we get that the only clean compact pattern of $\Theta_{e}(n)$ obyained in this way is $1^{\bullet}\in\hat{B}_{k}^{(prd)}$. The same happens if we choose to remove $1^{+}$. 3. 3.

Finally, if we remove the first element $n^{\bullet}$, we obtain $(n-2)^{\bullet}\dots 4^{\bullet}2^{\bullet}1^{+}3^{\bullet}\dots(n-3)^{\bullet}(n-1)^{\bullet}$. Notice that such a permutation ends with its maximum $(n-1)^{\bullet}$, so it has the same distance as $(n-2)^{\bullet}\dots 4^{\bullet}2^{\bullet}1^{+}3^{\bullet}\dots(n-3)^{\bullet}=\Theta_{e}(n-2)$ and we can conclude using the induction hypothesis.

In a completely analogous way, we can show that $\Lambda_{e}(n),\Theta_{o}(n)$ and $\Lambda_{o}(n)$ are minimal clean compact peg permutations in the complement of $\hat{B}_{k}^{(prd)}$ as well.

To conclude the proof, we now have to show that each of the exceptional permutations has distance exactly $n=k+2$. We already know that such a distance is at least $n$, as a consequence of Lemma 3.4. Regarding the opposite inequality, consider for instance $\Theta_{e}(n)$. Applying $2$ suitable prefix reversals to $\Theta_{e}(n)$, we obtain the peg permutation $\Theta_{e}(n-2)$ followed by $(n-1)^{\bullet}n^{\bullet}$. This is shown below:

[TABLE]

Notice that $prd(\hat{\Theta})\leq prd(\Theta_{e}(n-2))$, so, using the induction hypothesis:

[TABLE]

as desired. The same inequality can be proved in the same way for $\Theta_{o}(n)$. Finally, consider the permutation $\Lambda_{e}(n)$. Using a similar approach, we can apply a suitable prefix reversal as follows:

[TABLE]

the permutation $\hat{\Lambda}$ starts with the strip $(t+1)^{-}t^{\bullet}$, so, compacting and rescaling, we obtain the peg permutation $t^{-}(t+1)^{\bullet}(t-1)^{\bullet}(t+2)^{\bullet}\dots(n-2)^{\bullet}2^{\bullet}(n-1)^{\bullet}1^{\bullet}=\Lambda_{o}(n-1)$. Then, using again the induction hypothesis:

[TABLE]

The case $\Lambda_{o}(n)$ can be dealt with in the same way. $\blacksquare$

Corollary 3.6

• (i)

If $n=k+2$ is even, then $\Theta_{e}(n)$ and $\Lambda_{e}(n)$ are clean compact peg basis permutations for both $\hat{B}_{k}^{(prd)}$ and $\hat{B}_{k+1}^{(prd)}$.

• (ii)

If $n=k+2$ is odd, then $\Theta_{o}(n)$ and $\Lambda_{o}(n)$ are clean compact peg basis permutations for both $\hat{B}_{k}^{(prd)}$ and $\hat{B}_{k+1}^{(prd)}$.

Lemma 3.7

Let $\pi^{\varepsilon}$ be a clean compact peg basis permutation of $\hat{B}_{k}^{(prd)}$ of length $k+2$. Suppose $\beta^{b}$ is obtained from $\pi^{\varepsilon}$ by removing an element and suitably rescaling the remaining ones. If $\beta^{b}$ is clean compact, then $\beta_{k+1}^{b_{k+1}}=(k+1)^{+/\bullet}$.

Proof.  Let $\pi^{\varepsilon}=\pi_{1}^{\varepsilon_{1}}\cdots\pi_{k+2}^{\varepsilon_{k+2}}$. By the analogue of Lemma 2.11 in the case of prefix reversal, $\pi_{k+2}^{\varepsilon_{k+2}}\neq(k+2)^{+/\bullet}$. Moreover, by minimality of $\pi^{\varepsilon}$, it is $\beta^{b}\in\hat{B}_{k}^{(prd)}$, so $\beta^{b}=\alpha^{a}\cdot[v]_{peg}$ for some $k$-generating permutation $\alpha^{a}$ and some inflation vector $v$. Notice that $\alpha^{a}$ and $\beta^{b}$ have the same length $k+1$ and $\beta^{b}$ is clean compact, hence $v_{j}=1$ for each $j=1,\ldots,k+1$. Moreover, since it is easy to show that the last element of any generating permutation of $\hat{B}_{k}^{(prd)}$ is its maximum decorated $+$, we have that $\beta_{k+1}^{b_{k+1}}=(k+1)^{+/\bullet}$, as desired. $\blacksquare$

Theorem 3.8

Every clean compact peg basis permutation of $\hat{B}_{k}^{(prd)}$ has length at most $k+2$.

Proof.  The proof is identical to the first part of Theorem 2.12.

Theorem 3.9

Let $n=k+2$. Every clean compact peg basis permutation of $\hat{B}_{k}^{(prd)}$ has length at most $k+1=n-1$, apart from the exceptional ones $\Theta_{e}(n),\Theta_{o}(n),\Lambda_{e}(n)$ and $\Lambda_{o}(n)$.

Proof.  Suppose that $\pi^{\varepsilon}=\pi_{1}^{\varepsilon_{1}}\cdots\pi_{n}^{\varepsilon_{n}}$ is a clean compact basis permutation of $\hat{B}_{k}^{(prd)}$ of length $n=k+2$. By Lemma 2.10, we know that it is possible to remove one element of $\pi^{\varepsilon}$ in such a way that the resulting peg permutation $\beta^{b}$ is clean compact and thus, by the previous lemma, we have that $\beta_{n-1}^{b_{n-1}}=(n-1)^{+/\bullet}$. In particular, since $\pi_{n}^{\varepsilon_{n}}\neq n^{+/\bullet}$, $\beta^{b}$ can be obtained from $\pi^{\varepsilon}$ in one of the following (mutually exclusive) ways:

$\pi^{\varepsilon}$ ends with $n^{+/-/\bullet}(n-1)^{+/-/\bullet}$ and $\beta^{b}$ is obtained by removing one of the last two elements; 2. 2.

the last element of $\pi^{\varepsilon}$ is $(n-1)^{+/\bullet}$ and $\beta^{b}$ is obtained by removing $n^{+/-/\bullet}$, which is not the second to last element; 3. 3.

the second to last element of $\pi^{\varepsilon}$ is $n^{+/\bullet}$ and $\beta^{b}$ is obtained by removing the last element, which is not $(n-1)^{+/-/\bullet}$.

Our goal is to show that we can almost always remove another element from $\pi^{\varepsilon}$ to obtain a clean compact peg permutation that does not end with its maximum, which is a contradiction with Lemma 3.7. The only exceptions are the exceptional permutations of Theorem 3.5.

Suppose that $\pi^{\varepsilon}=\rho^{p}n^{+/-/\bullet}(n-1)^{+/-/\bullet}$, for some peg permutation $\rho^{p}$. Notice that $\rho^{p}$ has to be clean compact, because $\pi^{\varepsilon}$ is clean compact and its last two elements are the largest elements of the permutation, so no new strip can be created by removing any of them. Then, again by Lemma 2.10, we can remove an element of $\rho^{p}$ in order to obtain a clean compact peg permutation $\hat{\rho}^{\hat{p}}$. Thus the peg permutation $\hat{\rho}^{\hat{p}}(n-1)^{+/-/\bullet}(n-2)^{+/-/\bullet}$ is clean compact, has length $n-1$ and does not end with its maximum, as desired. 2. 2.

In the second case, we can remove the last element $(n-1)^{+/\bullet}$ in most of the situations, except when we have either $(n-2)^{+/\bullet}n^{+/\bullet}$ or $n^{-/\bullet}(n-2)^{-/\bullet}$.

If $\pi^{\varepsilon}$ contains $(n-2)^{+/\bullet}n^{+/\bullet}$, we can remove $(n-2)^{+/\bullet}$ to get a clean compact peg permutation, unless $(n-3)^{+/\bullet}$ is immediately before $(n-1)^{+/\bullet}$ in $\pi^{\varepsilon}$. By iterating this argument, we find out that we are always able to remove an element other than $n^{+/\bullet}$, except when $\pi^{\varepsilon}=2^{+/\bullet}4^{+/\bullet}6^{+/\bullet}\cdots n^{+/\bullet}1^{+/\bullet}3^{+/\bullet}5^{+/\bullet}\cdots(n-1)^{+/\bullet}$, if $n$ is even, and when $\pi^{\varepsilon}=1^{+/\bullet}3^{+/\bullet}5^{+/\bullet}\cdots n^{+/\bullet}2^{+/\bullet}4^{+/\bullet}6^{+/\bullet}\cdots(n-1)^{+}$, if $n$ is odd. In such cases we can remove the element $1^{+/\bullet}$ thus obtaining a clean compact peg permutation that does not end with its maximum, as desired.

Otherwise, if $\pi^{\varepsilon}$ contains $n^{-/\bullet}(n-2)^{-/\bullet}$, we can try to remove $(n-2)^{-/\bullet}$ and argue analogously, obtaining the “bad” peg permutations $n^{-/\bullet}(n-2)^{-/\bullet}\dots 4^{-/\bullet}2^{-/\bullet}1^{+/\bullet}3^{+/\bullet}\dots(n-3)^{+/\bullet}(n-1)^{+/\bullet}$, if $n$ is even and $t=\frac{n}{2}$, and $n^{-/\bullet}(n-2)^{-/\bullet}\dots 3^{-/\bullet}1^{-/\bullet}2^{+/\bullet}4^{+/\bullet}\dots(n-3)^{+/\bullet}(n-1)^{+/\bullet}$, if $n$ is odd and $t=\frac{n+1}{2}$. This time it is impossible to find an element that can always be removed in order to find a clean compact peg permutation. Notice that we have to decorate $1$ with $+$, if $n$ is even, and with $-$, if $n$ is odd. If we decorate the other elements in the “minimal” way, i.e. using $\bullet$, we obtain, respectively, $\Theta_{e}(n)$ if $n$ is even and $\Theta_{o}(n)$ if $n$ is odd, as desired. 3. 3.

The third case is somehow symmetric to the second one. Proceeding in an analogous way (just starting with the removal of the second to last element of $\pi^{\varepsilon}$), we can find a clean compact peg permutation having the required properties, except for the permutations $(t+1)^{+/\bullet}t^{-/\bullet}(t+2)^{+/\bullet}(t-1)^{-/\bullet}\dots(n-1)^{+/\bullet}2^{-/\bullet}n^{+/\bullet}1^{-/\bullet}$, if $n$ is even and $t=\frac{n}{2}$, and $t^{-/\bullet}(t+1)^{+/\bullet}(t-1)^{-/\bullet}(t+2)^{+/\bullet}\dots(n-1)^{+/\bullet}2^{-/\bullet}n^{+/\bullet}1^{-/\bullet}$, if $n$ is odd and $t=\frac{n+1}{2}$. This time we have that $t+1$ has to be decorated with $+$, if $n$ is even, and $t$ with $-$, if $n$ is odd. Decorating the remaining elements with $\bullet$, as above, we obtain $\Lambda_{e}(n)$ and $\Lambda_{o}(n)$, respectively. $\blacksquare$

As an example, if we choose $k=3$, then we have that the exceptional permutations $\Theta_{o}(5)=5^{\bullet}3^{\bullet}1^{-}2^{\bullet}4^{\bullet}$ and $\Lambda_{o}(5)=3^{-}4^{\bullet}2^{\bullet}5^{\bullet}1^{\bullet}$ are clean compact peg basis permutations for both $\hat{B}_{3}^{(prd)}$ and $\hat{B}_{4}^{(prd)}$; in fact, they have distance $5$ and all their clean compact patterns have distance at most $3$. Notice that also $\Theta_{e}(4)$ and $\Lambda_{e}(4)$ are clean compact peg basis permutations for $\hat{B}_{3}^{(prd)}$.

Regarding the (standard) basis of $B_{k}^{(prd)}$, all the results presented in the previous section can be adapted. The main difference is that, in the case of the prefix reversal distance, there are clean compact peg basis permutations of $\hat{B}_{k}^{(prd)}$ that have distance $k+2$, so there can be (even large) gaps between their distance and the distance of their clean compact patterns. For example, this is true for the exceptional permutations of Theorem 3.5. In fact, we can say something more precise about the basis elements that derive from the exceptional permutations.

Theorem 3.10

For a given peg permutation $\pi^{\varepsilon}$, consider the set:

[TABLE]

Also, denote with $min(\pi^{\varepsilon})$ the (standard) permutation obtained by inflating each element of $\pi^{\varepsilon}$ decorated $\bullet$ by $1$, and the remaining ones by $2$. Then, for each $n$:

if $n$ is even, $\mathcal{M}_{\Theta_{e}(n)}=\left\{min(\Theta_{e}(n))\right\}$ and $\mathcal{M}_{\Lambda_{e}(n)}=\left\{min(\Lambda_{e}(n))\right\}$; in particular, $min(\Theta_{e}(n))$ and $min(\Lambda_{e}(n)$ are basis permutations for $B_{n-1}^{(prd)}$. 2. 2.

If $n$ is odd, $\mathcal{M}_{\Theta_{o}(n)}=\left\{min(\Theta_{o}(n))\right\}$ and $\mathcal{M}_{\Lambda_{o}(n)}=\left\{min(\Lambda_{o}(n))\right\}$; in particular, $min(\Theta_{o}(n))$ and $min(\Lambda_{o}(n))$ are basis permutations for $B_{n-1}^{(prd)}$.

Proof. (sketch) First of all, notice that, in general, $peg(min(\pi^{\varepsilon}))=\pi^{\varepsilon}$ and $min(\pi^{\varepsilon})$ is the minimal permutation with this property (with respect to the pattern involvement relation). We have $\gamma:=min(\Theta_{e}(n))=(n+1)\ (n-1)\dots 5\ 3\ 1\ 2\ 4\ 6\dots(n-2)\ n$. We wish to show that $prd(\gamma)=prd(\Theta_{e}(n))=n$, which implies that $\mathcal{M}_{\Theta_{e}(n)}=\left\{\gamma\right\}$, and that each pattern of $\gamma$ has distance at most $n-1$; this guarantees that $\gamma$ is a basis permutation. We give the idea of the proof for $\Theta_{e}(n)$, the other cases being similar.

Define an adjacency of $\gamma$ as a consecutive pair $\gamma_{i}\gamma_{i+1}$ in $\gamma$ such that $\gamma_{i+1}=\gamma_{i}+1$. Then, by adapting Lemma 3.7 to non-peg permutations, using adjacencies in place of breakpoints, and observing that $\gamma$ does not end with its maximum, we can prove that $prd(\gamma)\geq n$. Conversely, we can mimick the sorting procedure used in Theorem 3.5 for $\Theta_{e}(n)$ to obtain an analogous sorting sequence of length $n$ for $\gamma$, so $prd(\gamma)\leq n$. Therefore we have $prd(\gamma)=n$ and $\gamma$ is the minimal permutation such that $peg(\gamma)=\Theta_{e}(n)$, so $\mathcal{M}_{\Theta_{e}(n)}=\left\{\gamma\right\}$, as desired.

Finally, in analogy with the proof of Theorem 3.5, a simple case by case analysis shows that each pattern of $\gamma$ can be sorted using at most $n-1$ prefix reversals, meaning that $\gamma$ belongs to the basis of $B_{n-1}^{(prd)}$. $\blacksquare$

As a consequence of the previous theorem, we have that the exceptional permutations of length $k+2$ are clean compact peg basis permutations of both $\hat{B}_{k}^{(prd)}$ and $\hat{B}_{k+1}^{(prd)}$, but they contribute to the standard basis just for $\hat{B}_{k+1}^{(prd)}$. For example, if $k$ is even, then we proved that $min(\Theta_{o}(k+1))$ and $min(\Lambda_{o}(k+1))$ are basis permutations for $B_{k}^{(prd)}$; moreover, also $\Theta_{e}(k+2)$ and $\Lambda_{e}(k+2)$ are in the clean compact peg basis, but $min(\Theta_{e}(n-2))$ contains $min(\Theta_{o}(k+1))$ as a pattern, so it cannot belong to the basis; analogously, $min(\Lambda_{o}(k+2)\geq min(\Lambda_{o}(k+1)$. Consider for instance the ball $\hat{B}_{1}^{(prd)}$. Theorem 3.9 and a direct computation guarantee that its clean compact peg basis is the set $\{1^{\bullet}2^{-},2^{\bullet}1^{+},2^{+}1^{\bullet}\}$ together with the exceptional permutations $\Theta_{o}(3)=3^{\bullet}1^{-}2^{\bullet}$ and $\Lambda_{o}(3)=2^{-}3^{\bullet}1^{\bullet}$. Observe that $2^{\bullet}1^{+}=\Theta_{e}(2)$ and $2^{+}1^{\bullet}=\Lambda_{e}(2)$ are the exceptional permutations of length $2$. The corresponding sets of candidate basis permutations are $\mathcal{M}_{1^{\bullet}2^{-}}=\left\{132\right\}$, $\mathcal{M}_{2^{+}1^{\bullet}}=\left\{231\right\}$, $\mathcal{M}_{2^{\bullet 1^{+}}}=\left\{312\right\}$, $\mathcal{M}_{3^{\bullet}1^{-}2^{\bullet}}=\{4213\}$ and $\mathcal{M}_{2^{-}3^{\bullet}1^{\bullet}}=\{3241\}$; since $min(\Theta_{e}(2))=312\leq 4213=min(\Theta_{o}(3))$, $min(\Lambda_{e}(2))=231\leq 3241=min(\Lambda_{o}(3))$, the basis of $\hat{B}_{1}^{(prd)}$ is $\left\{132,231,312\right\}$.

Using the same approach, but with considerably more efforts, a tedious computation shows that the basis of $B_{2}^{(prd)}$ is $\left\{132,3241,3412,4213,4231\right\}$. Together with the results of [HV16], this implies that $|Av(132,3241,3412,4213,4231)|=B_{2}^{(prd)}=n^{2}+1$ (sequence A002522 in Sloane’s Encyclopedia).