Enumerations of some pattern-avoiding Fishburn permutations

TL;DR

This paper proves conjectures related to counting specific classes of pattern-avoiding Fishburn permutations, including those avoiding pattern 321 and certain classical patterns of sizes 4 and 5.

Contribution

It confirms two conjectures by Egge and provides enumeration results for Fishburn permutations avoiding particular classical patterns.

Findings

Enumeration of Fishburn permutations avoiding pattern 321 and certain classical patterns.

Proof of two conjectures of Egge regarding pattern-avoiding Fishburn permutations.

Enumeration formulas for these classes of permutations.

Abstract

In this paper, we prove two conjectures of Egge on the enumeration of several classes of pattern-avoiding Fishburn permutations. Our results include enumerating Fishburn permutations avoiding pattern 321 and one of the following three types of classical patterns: a pattern of size 4, two patterns of size 4, or a pattern of size 5.

Tables

Table 1. Table 1: Enumeration of σ 𝜎 \sigma -avoiding Fishburn permutations

Patterns ${\sigma }_1,\cdots ,{\sigma }_k$	|$F_n({\sigma }_1,\cdots ,{\sigma }_k)$|	Reference
321, 1243	$n^2-3n+4\left(n\ge 2\right)$	Theorem 2.1
321, 2134	$n^2-3n+4\left(n\ge 2\right)$	Theorem 2.4
321, 1324	$\frac{3}{2}{n}^2-\frac{13}{2}n+10\left(n\ge 3\right)$	Theorem 2.7
321, 1423, 2143	$\left(\frac{n}{2}\right)+1\left(n\ge 0\right)$	Theorem 3.1
321, 3142, 2143	$\left(\frac{n}{2}\right)+1\left(n\ge 0\right)$	Theorem 3.4
321, 2143, 3124	$\left(\frac{n}{2}\right)+1\left(n\ge 0\right)$	Theorem 3.7
321, 2143, 4123	$\left(\frac{n}{2}\right)+1\left(n\ge 0\right)$	Theorem 3.8
321, 1423, 3124	$F_n+2\left(n\ge 4\right)$	Theorem 3.11
321, 1423, 4123	$F_{n+1}-1\left(n\ge 1\right)$	Theorem 3.14
321, 3124, 4123	$F_{n+1}-1\left(n\ge 1\right)$	Theorem 3.17
321, 14253	$2^n-\left(\frac{n}{2}\right)-1\left(n\ge 1\right)$	Theorem 4.1
321, 21354	$2^n-\left(\frac{n}{2}\right)-1\left(n\ge 1\right)$	Theorem 4.3
321, 3142	$2^{n-1}\left(n\ge 1\right)$	Lemma 4.2
321,132	$n\left(n\ge 1\right)$	Lemma 4.4

Equations

$$1+\sum _{n\ge 1}\prod _{i=1}^n(1-(1-t{)}^i).$$

$${\pi }^{\prime }=1k{}^{1}2{}^2\cdots {}^{k-2}k-1{}^{k-1}$$

$$\pi =1k2\cdots k-1k+1\cdots n.$$

$$|F_n^{(1)}(321,1243)|=1+(n-3)+1=n-1,$$

$$|F_4^{(2)}(321,1243)|=5=4^2-4\times 4+5.$$

$${\pi }^{\prime }=n-112\cdots n-2$$

$$\pi =k12\cdots k-1k+1\cdots n.$$

$$k1k+1{}^{1}k+2{}^{2}k+3{\cdots }^{n-k-1}n{}^{n-k}.$$

$$\pi =k1{\pi }_{3}2\cdots k-1k+1\cdots {\pi }_3-1{\pi }_3+1\cdots n.$$

$$1+(n-k)+(n-k-1)=2(n-k).$$

$$|F_n^{(2)}(321,1243)|=1+2+\sum _{k=3}^{n-2}2(n-k)+(n-2)=n^2-4n+5.$$

$$|F_n(321,1243)|=$$

$$=$$

$${\pi }^{\prime }=n-21{}^{1}2{}^2\cdots {}^{n-4}n-3{}^{n-3}.$$

$${\pi }^{\prime \prime }=n-21{}^{1}2{}^2\cdots {}^{n-4}n-3{}^{n-3}n-1.$$

$$|F_n^{(2)}(321,2134)|=1+(n-2)+(n-3)=2n-4.$$

$${\pi }^{\prime }=1\cdots jn-1{}^{1}j+1{}^2\cdots {}^{n-j-2}n-2{}^{n-j-1}$$

$$\sum _{j=1}^{n-3}(n-j-2)=\sum _{j=1}^{n-3}j=\frac{1}{2}{n}^2-\frac{5}{2}n+3.$$

$${\pi }^{\prime }=1\cdots jn-2j+1{}^1\cdots {}^{n-j-2}n-3{}^{n-j-3}.$$

$$\sum _{j=1}^{n-4}(n-j-3)=\sum _{j=1}^{n-4}j=\frac{1}{2}{n}^2-\frac{7}{2}n+6.$$

$$|F_n^{(1)}(321,2134)|=$$

$$=$$

$$|F_2(321,2134)|=2=2^2-3\times 2+4.$$

$$|F_n(321,2134)|=$$

$$=$$

$${\pi }^{\prime }=1{}^{1}2{}^2\cdots {}^{n-2}n-1{}^{n-1}.$$

$$|F_n^{(1)}(321,1324)|=n-1.$$

$${\pi }^{\prime }=21{}^{1}3{}^2\cdots {}^{n-3}n-1{}^{n-2}$$

$$\pi =n12\cdots n-1.$$

$${\pi }^{\prime }=n-11{}^{1}2{}^2{\cdots }^{n-3}n-2{}^{n-2}.$$

$${\pi }^{\prime }=n-21{}^{1}2{}^2{\cdots }^{n-4}n-3{}^{n-3}.$$

Full text

**Enumerations of some pattern-avoiding Fishburn permutations **

Yujie Du and Philip B. Zhang111 Email: zhang@tjnu.edu.cn This work was supported by the National Science Foundation of China under Grant No. 12171362.

College of Mathematical Science

Tianjin Normal University, Tianjin 300387, P. R. China

Abstract. In this paper, we prove several enumeration results of pattern-avoiding Fishburn permutations that Egge proposed recently. Our results include enumerating Fishburn permutations avoiding 321 and one of the following three types of patterns: a classical pattern of size 4, two classical patterns of size 4, or a classical pattern of size 5.

Keywords: Fishburn permutations; Fishburn numbers; Pattern avoidance; Fibonacci numbers

AMS Classification 2020: 05A15

1 Introduction

The purpose of this paper is to prove some enumeration formulae, conjectured by Egge [3], of pattern-avoiding Fishburn permutations.

Enumerations of pattern-avoiding permutations have attracted lots of interest. Recall that the reduced form of a permutation on the set $\{j_{1},j_{2},\cdots,j_{r}\}$ with $j_{1}<j_{2}<\cdots<j_{r}$ is a rearrangement of $1\ 2\cdots r$ by replacing $j_{i}$ with $i$ for $1\leq i\leq r$ without changing the relative order in the permutation. For example, the reduced form of permutation 342 is 231. We say a permutation $\pi$ containing a classical pattern $\tau$ if $\pi$ has a subsequence (of the same length as $\tau$) whose reduced form is $\tau$. A permutation $\pi$ avoiding a pattern $\tau$ means $\pi$ does not contain $\tau$. We refer the readers to the book of Kitaev [6] for a comprehensive study.

Fishburn permutations are permutations avoiding the Fishburn pattern. A permutation $\pi=\pi_{1}\pi_{2}\cdots\pi_{n}$ contains a Fishburn pattern, denoted by $f$ (see Fig. 1), if $\pi$ has a subsequence $\pi_{i},\pi_{i+1},\pi_{j}$ with $1\leq i\leq n-2$ and $i+2\leq j\leq n$ such that $\pi_{i}<\pi_{i+1}$ and $\pi_{i}=\pi_{j}+1$.

Bousquet-Mélou, Claesson, Dukes, and Kitaev [2] gave bijections between permutations avoiding the pattern $f$ and ascent sequences, unlabeled (2+2)-free posets, and linearized chord diagrams. They also showed their generating function is

[TABLE]

Since the enumeration is related to the Fishburn numbers [10, 7, 1, 4], Gil and Weiner [5] call permutations avoiding the Fishburn pattern as Fishburn permutations.

We are concerned in this paper with the enumeration of Fishburn permutations which avoid some classical patterns. Gil and Weiner [5] studied the enumeration of Fishburn permutations which avoid classical patterns of size 3 or 4 in both regular and indecomposable forms. Based on Gil and Weiner’s work, Egge [3] settled one of their conjectures and further studied the enumeration of Fishburn permutations that avoid classical pattern sets of size 3 or 4 by using generating tree techniques. Furthermore, he concluded several conjectures of the pattern avoidance of Fishburn permutations.

Let $\sigma_{1},\cdots,\sigma_{k}$ be some classical patterns. Denote by $F_{n}(\sigma_{1},\cdots,\sigma_{k})$ the set of Fishburn permutations of length $n$ which avoid each $\sigma_{1},\cdots,\sigma_{k}$. We prove an enumeration conjecture of Egge [3, Conjecture 10.17]. Table 1 is a summary of main results of this paper.

The following lemma tells that all the enumeration results discussed in this paper can be classified into two types according to where the smallest letter 1 is placed.

Lemma 1.1**.**

Let $\pi\in F_{n}(321)$. Then either $\pi_{1}=1$ or $\pi_{2}=1$.

Proof.

We shall prove this lemma by contradiction. Suppose the letter 1 does not stay in the first two sites. Let $\pi_{1}=x$ and $\pi_{2}=y$. If $x>y$, then $x,y$, and $1$ would be a copy of 321. If $x<y$, then $x-1$ must appear to the right of $y$. Therefore, the letters $x,y$, and $x-1$ would be a copy of the Fishburn pattern $f$. This leads to a contradiction. Thus we have $\pi_{1}=1$ or $\pi_{2}=1$. ∎

As the enumerations in this paper all involve 321-avoiding Fishburn permutations, we classify the proofs of the enumeration results into two types according to Lemma 1.1 and enumerate them separately. Throughout the paper we shall denote the set $\{\pi\in F_{n}(321,\sigma_{1},\cdots,\sigma_{k}):\pi_{i}=1\}$ by $F_{n}^{(i)}(321,\sigma_{1},\cdots,\sigma_{k})$ for $i=1$ or 2.

The rest of this paper is organized as follows. In Section 2, we prove three enumeration results, Theorems 2.1, 2.4, and 2.7, of Fishburn permutations which avoid 321 and a classical pattern of size 4. In Section 3, we consider the enumeration of Fishburn permutations that avoid 321 and two classical patterns of size 4. It is worth mentioning that Theorems 3.11, 3.14, and 3.17 involve the famous Fibonacci numbers. In Section 4, the enumeration results involve pattern avoidance of size 5. We remark that the last two lines of Table 1 have not been found or conjectured before, which will be used to prove Theorems 4.1 and 4.3.

2 Avoiding 321 and a classical pattern of size 4

2.1 Enumerating $F_{n}(321,1243)$

The main result of this subsection is as follows.

Theorem 2.1**.**

For $n\geq 2$, $|F_{n}(321,1243)|=n^{2}-3n+4$.

The enumeration of $F_{n}(321,1243)$ can be divided into the following two propositions according to the site where 1 is located in the permutation. The proof of Theorem 2.1 will be given at the end of this subsection after demonstrating the following two propositions.

Proposition 2.2**.**

For $n\geq 2$, $|F_{n}^{(1)}(321,1243)|=n-1$.

Proof.

For $n=2$, $F_{2}^{(1)}(321,1243)=\{12\}$. Thus $|F_{2}^{(1)}(321,1243)|=1=2-1.$

For $n=3$, $F_{3}^{(1)}(321,1243)=\{123,132\}$. Thus $|F_{3}^{(1)}(321,1243)|=2=3-1$.

Next consider $n\geq 4$. We classify $F_{n}^{(1)}(321,1243)$ into several cases according to the value of $\pi_{2}$.

1. (1)

$\pi_{2}=2$. To avoid 1243, the letters $3,\cdots,n$ are certainly in increasing order. In this case, $\pi=1\ 2\cdots n$. 2. (2)

$3\leq\pi_{2}=k\leq n-1$. First, avoiding 321 implies the letters $2,\cdots,k-1$ appearing after $k$ can only be increasing. Let

[TABLE]

be a permutation of $\{1,2,\ldots,k\}$, where we label the sites between two letters for our convenience of discussion. For any $x\in\{k+1,\cdots,n\}$, we claim that $x$ must be in $(k-1)$-th site. Otherwise, if $x$ is placed in the first site, then $k,x,k-1$ would be a copy of $f$; and if $x$ is in the $i$-th site with $2\leq i\leq k-2$, then $1,2,x,k-1$ would be a copy of 1243. Therefore, $k+1,\cdots,n$ must be in the rightmost site of $\pi^{\prime}$.

Next, to avoid 1243, the letters $k+1,\cdots,n$ appearing after $1\ k$ are also in increasing order. Hence $\pi$ is surely of the following form

[TABLE]

Because $k$ has $n-3$ possible values, the number of permutations of the above form is $n-3$. 3. (3)

$\pi_{2}=n$. The letters $2,\cdots,n-1$ are to the right of $n$, then the forbidden pattern 321 implies they are in increasing order. So $\pi=1\ n\ 2\cdots n-1$.

It follows that for $n\geq 4$,

[TABLE]

which completes the proof. ∎

Given a permutation $\pi\in F_{n-1}(\sigma)$, recall that a site of $\pi$ is active if the new permutation of length $n$ obtained by inserting $n$ into this site is also a Fishburn permutation avoiding pattern $\sigma$.

Proposition 2.3**.**

For $n\geq 2$, $|F_{n}^{(2)}(321,1243)|=n^{2}-4n+5$.

Proof.

For $n=2$, $F_{2}^{(2)}(321,1243)=\{21\}$. Thus $|F_{2}^{(2)}(321,1243)|=1=2^{2}-4\times 2+5$.

For $n=3$, $F_{3}^{(2)}(321,1243)=\{213,312\}$. Thus $|F_{3}^{(2)}(321,1243)|=2=3^{2}-4\times 3+5$.

For $n=4$, $F_{4}^{(2)}(321,1243)=\{2134,2143,3124,3142,4123\}$. Thus

[TABLE]

Next we shall prove for $n\geq 5$. Let $\pi_{1}=k$. We classify according to the value of $k$.

1. (1)

$k=n$. To avoid 321, the letters $2,\cdots,n-1$ appearing after $1$ must be in increasing order. So $\pi=n\ 1\ 2\cdots n-1$. 2. (2)

$k=n-1$. To avoid 321, the letters $2,\cdots,n-2$ can only be increasing. Suppose that

[TABLE]

and consider the number of active sites of inserting $n$. We assert there are two active sites: one is immediately to the right of 1, and the other one is the rightmost site of $\pi^{\prime}$. If $n$ was inserted to the sites except these two, then $1,2,n,n-2$ would create a copy of 1243. Thus either $\pi=n-1\ 1\ n\ 2\cdots n-2$ or $\pi=n-1\ 1\ 2\cdots n-2\ n$. Clearly, the number of permutations in such a case is 2. 3. (3)

$3\leq k\leq n-2$. Suppose that $\pi=k\ 1\ \pi_{3}\pi_{4}\cdots\pi_{n}$. Next we proceed to consider the value of $\pi_{3}$.

1. (i)

$\pi_{3}<k$. We claim $\pi_{3}=2$. Otherwise, $k,\pi_{3},2$ would be a copy of 321. Moreover, to avoid 1243, the letters $3,\cdots,k-1,k+1,\cdots,n$ to the right of 1 2 are certainly in increasing order. So

[TABLE] 2. (ii)

$\pi_{3}=k+1$. First, the letters $2,\cdots,k-1$ are in increasing order in the permutation to avoid 321. Second, $k+2,\cdots,n$ are also in increasing order. If not, $1,k+1$ and the decreasing pair would be a copy of 1243. Third, no letters in $k+2,\cdots,n$ can be inserted into $2\cdots k-1$. If $x\in\{k+2,\cdots,n\}$ is inserted into the sites between $2\cdots k-1$, say $x$ is inserted between $i$ and $i+1$ ($2\leq i\leq k-2$), then $1,i,x,i+1$ would create a copy of 1243. Now, we just need to consider $2\cdots k-1$ as a whole part and insert it into

[TABLE]

Indeed, as labeled above, the $n-k$ sites are all available to insert $2\cdots k-1$. So the number of permutations in subcase (ii) is $n-k$. 3. (iii)

$\pi_{3}\geq k+2$. We claim $\pi_{4}=2$. If $2<\pi_{4}<\pi_{3}$, then $\pi_{3},\pi_{4},2$ would be a copy of 321. Whereas if $\pi_{4}>\pi_{3}$, then $\pi_{3},\pi_{4},\pi_{3}-1$ would create a copy of $f$. Similar to the discussion in the above paragraph, $2,\cdots,k-1$ are in increasing order and no letters in $\{k+1,\cdots,\pi_{3}-1,\pi_{3}+1,\cdots,n\}$ can be inserted into the sites $2,\cdots,k-1$ formed to avoid 1243. Thus $k+1,\cdots,\pi_{3}-1,\pi_{3}+1,\cdots,n$ are in increasing order to the right of $k-1$. So

[TABLE]

Because of $\pi_{3}$ ranging from $k+2$ to $n$, the number of permutations in this subcase is $n-k-1$.

For a fixed $k$, the number of permutations of the above three forms in Case (3) is

[TABLE] 4. (4)

$k=2$. We shall classify according to the value of $\pi_{3}$.

1. (i)

$\pi_{3}=3$. The forbidden pattern 1243 indicates that $4,\cdots,n$ appearing after 1 3 must be in increasing order. So $\pi=2\ 1\ 3\ 4\cdots n$. 2. (ii)

$\pi_{3}>3$. Suppose that $\pi=2\ 1\ x\ y\cdots$, where $x>3$. We first claim $y=3$. If $y>x$, then $x,y,x-1$ is a copy of $f$. If $3<y<x$, then $x,y,3$ is a copy of 321. So $y=3$. Moreover, the letters $4,\cdots,x-1,x+1,\cdots,n$ must be in increasing order since they appear to the right of 1 and 3 to avoid 1243.

Indeed, the number of permutations in Case (4) depending on $\pi_{3}$ is $n-2$.

Therefore, for $n\geq 5$,

[TABLE]

We have thus proved this proposition. ∎

Proof of Theorem 2.1.

For $n\geq 2$, we obtain

[TABLE]

∎

2.2 Enumerating $F_{n}(321,2134)$

The main result of this subsection is as follows.

Theorem 2.4**.**

For $n\geq 2$, $|F_{n}(321,2134)|=n^{2}-3n+4$.

Proposition 2.5**.**

For $n\geq 3$, $|F_{n}^{(2)}(321,2134)|=2n-4$.

Proof.

For $n=3$, $|F_{3}^{(2)}(321,2134)|=2=2\times 3-4$.

For $n\geq 4$, we discuss the value of $\pi_{1}$. We claim $\pi_{1}\geq n-2$ when $\pi_{2}=1$. If the claim is not true, then $\pi_{1}=k\leq n-3$. To avoid 2134, the letters $k+1,\cdots,n$ appearing after $k\ 1$ must be in decreasing order. In fact, since $|\{k+1,\cdots,n\}|=n-k\geq 3$, there surely exists a forbidden copy of 321, which $\pi$ avoids. Thus $k\geq n-2$ as claimed. The proposition holds for $n=3$ is easy to check. Next we proceed to discuss according to the three values of $k$ for $n\geq 4$.

1. (1)

$k=n$. Undoubtedly, the letters $2,\cdots,n-1$ which are to the right of $n$ must be increasing to avoid 321. So $\pi=n\ 1\ 2\cdots n-1$. 2. (2)

$k=n-1$. Similarly, $2,\cdots,n-2$ to the right of $n-1$ are in increasing order. Suppose that $\pi^{\prime}=n-1\ 1\ ^{1}\ 2\ ^{2}\cdots\ ^{n-3}\ n-2\ ^{n-2}$, and we just need to insert $n$ to create $\pi$. It is trivial to see that all the $n-2$ sites are active. So the number of permutations in this case is $n-2$. 3. (3)

$k=n-2$. To avoid 321, the letters $2,\cdots,n-3$ are in increasing order. Let

[TABLE]

We just need to insert $n-1$ and $n$ into the sites labeled from 1 to $n-3$ to get $\pi$. First, $n$ is surely to the left of $n-1$ to avoid 2134. Second, $n-1$ can not appear in the $i$-th site for $1\leq i\leq n-4$ because $n,n-1,n-3$ would create a copy of 321. So $n-1$ can only be inserted to the $(n-3)$-th site. Further, let

[TABLE]

We should insert $n$ into $\pi^{\prime\prime}$ to obtain $\pi$. Noticed that $n$ can be inserted into all the $n-3$ sites. The number of permutations in this case is $n-3$.

Therefore, for $n\geq 4$,

[TABLE]

∎

Proposition 2.6**.**

For $n\geq 3$, $|F_{n}^{(1)}(321,2134)|=n^{2}-5n+8$.

Proof.

For $n=3$, $|F_{3}^{(1)}(321,2134)|=2=3^{2}-5\times 3+8$.

For $n\geq 4$, we discuss the value of $\pi_{2}$. Let $\pi_{2}=k$. We shall prove $k=2$ or $k\geq n-2$ by contradiction. Suppose $3\leq k\leq n-3$. First, to avoid 321, the letters $2,\cdots,k-1$ are in increasing order. Second, the Fishburn condition forces $\pi_{3}=2$, otherwise $k,\pi_{3},k-1$ would create a copy of $f$. Furthermore, the forbidden pattern 2134 implies that $k+1,\cdots,n$ to the right of $k\ 2$ must be in decreasing order. However, under our assumption of $k\leq n-3$, we have $|\{k+1,\cdots,n\}|=n-k\geq 3$ and we can surely find a copy of 321, a contradiction. So $k=2$ or $k\geq n-2$.

Suppose $\pi=1\ 2\cdots j\ x\cdots$, where $1\leq j\leq x-2$ and $n-2\leq x\leq n$. Notice $\pi=1\ 2\cdots n$ is an exceptional permutation in $F_{n}(321,2134)$. Next our discussion is divided into three cases according to the value of $x$.

1. (1)

$x=n$. Avoiding 321 implies that $j+1,\cdots,n-1$ to the right of $n$ must be increasing. So $\pi=1\cdots j\ n\ j+1\cdots n-1$, in which $j$ ranges from 1 to $n-2$. The number of permutations in this case is $n-2$. 2. (2)

$x=n-1$. Then $j+1,\cdots,n-2$ to the right of $n-1$ are also increasing to avoid 321. Suppose that

[TABLE]

and insert $n$ into $\pi^{\prime}$. Because $n-1,n,n-2$ is a forbidden copy of $f$, the letter $n$ can not be inserted into the first site. Actually it can be inserted into the $i$-th site for $2\leq i\leq n-j-1$, totally $n-j-2$ active sites. At this time $j$ ranges from 1 to $n-3$. So the number of permutations in this case is

[TABLE] 3. (3)

$x=n-2$. First, the letters $j+1,\cdots,n-3$ appearing after $n-2$ are in increasing order as $\pi$ avoids 321. Suppose that

[TABLE]

Second, we consider into which sites we can insert $n$ and $n-1$. We claim $n$ and $n-1$ can not be located in the site immediately to the right of $n-2$, since either $n-2,n,n-3$ or $n-2,n-1,n-3$ is a copy of $f$. Furthermore, $n$ must be to the left of $n-1$. If not, $n-2,j+1,n-1,n$ would be a copy of 2134. Next we claim $n-1$ can only be to the right of $n-3$. Otherwise, $n,n-1,n-3$ would be a copy of 321. Thus $n-1$ can only be in the $(n-j-3)$-th site, and it is easy to check that $n$ can be either in the site labeled from 1 to $n-j-2$ or the site immediately to the left of $n-1$. There are $n-j-3$ ways to place $n$ and $n-1$. In view of $j$ ranging from 1 to $n-4$, the number of permutations in this case is

[TABLE]

Therefore, for $n\geq 4$,

[TABLE]

∎

Proof of Theorem 2.4.

For $n=2$, $F_{n}(321,2134)=\{12,21\}$, so

[TABLE]

For $n\geq 3$, we get

[TABLE]

which completes the proof. ∎

2.3 Enumerating $F_{n}(321,1324)$

The main result of this subsection is as follows.

Theorem 2.7**.**

For $n\geq 3$, $|F_{n}(321,1324)|=\dfrac{3}{2}n^{2}-\dfrac{13}{2}n+10$.

Proposition 2.8**.**

For $n\geq 3$, $|F_{n}^{(1)}(321,1324)|=n-1$.

Proof.

We discuss according to the site where $n$ stays. To avoid 1324 and 321, we first show that there are no descents either to the left or to the right of $n$.

• •

If there is a decreasing pair $a,b$ to the left of $n$, then $1,a,b,n$ would be a copy of 1324.

• •

If $a,b$ is to the right of $n$, then $n,a,b$ would be a copy of 321.

We proceed to claim that the letters to the right of $n$ must be greater than those to the left of $n$. Otherwise, let $z$ be the largest letter to the left of $n$ and let $y$ be a letter to the right of $n$ smaller than $z$. The Fishburn condition forces $z-1$ must be immediately to the left of $z$. Similarly $z-2$ is immediately to the left of $z-1$ and so forth until $y+1$. Then $y+1,y+2,y$ would be a copy of $f$, which $\pi$ avoids. This proves our claim.

Therefore, all the letters except $n$ should be in increasing order. Now suppose that

[TABLE]

Consider where we can insert $n$ into $\pi^{\prime}$ to get $\pi$. Surely the number of permutations in this case is $n-1$.

Therefore, for $n\geq 3$,

[TABLE]

∎

Proposition 2.9**.**

For $n\geq 3$, $|F_{n}^{(2)}(321,1324)|=\dfrac{3}{2}n^{2}-\dfrac{15}{2}n+11$.

Proof.

We can check directly for $n\leq 5$. Suppose $\pi_{1}=k$. We classify according to the value of $k$ for $n\geq 6$.

1. (1)

$k=2$. The form of permutations in this case is similar to the one in Proposition 2.8, just see two letters $2\ 1$ as the smallest letter. Suppose that

[TABLE]

and insert $n$. Hence the number of permutations in this case is $n-2$. 2. (2)

$k=n$. The condition avoiding 321 means $2,\cdots,n-1$ to the right of $n$ are in increasing order. So

[TABLE] 3. (3)

$k=n-1$. Then we have $2,\cdots,n-2$ are in increasing order. Suppose that

[TABLE]

Next we will insert $n$ into $\pi^{\prime}$. We find that all the $n-2$ sites are active. So the number of permutations in this case is $n-2$. 4. (4)

$k=n-2$. The forbidden pattern 321 implies $2,\cdots,n-3$ to the right of $n-2$ are in increasing order. Suppose that

[TABLE]

and consider into which sites we can insert $n$ and $n-1$ to obtain $\pi$.

1. (i)

If $n$ is to the right of $n-1$, then $n$ must be adjacent to $n-1$. Otherwise let $x$ be a letter between them, then $1,n-1,x,n$ would be a copy of 1324, which $\pi$ avoids. Actually, we can consider $n-1\ n$ as a whole part and insert it into all the $n-3$ sites in $\pi^{\prime}$. The number of permutations in this subcase is $n-3$. 2. (ii)

If $n$ is to the left of $n-1$, then $n-1$ must be inserted into the rightmost site to avoid 321 and $n$ can be inserted into all the $n-3$ sites. The number of permutations in this subcase is $n-3$.

It follows that the number of permutations when $k=n-2$ is $2(n-3)$. 5. (5)

$3\leq k\leq n-3$. Avoiding 321 implies $2,\cdots,k-1$ to the right of $k\ 1$ must be in increasing order. Next we consider where $k+1,\cdots,n$ are placed.

1. (i)

None of the letters in $\{k+1,\cdots,n\}$ are to the left of $k-1$. Then we can consider $k\ 1\ 2\cdots k-1$ as the smallest letter 1. By Proposition 2.8, the number of permutations in this subcase is $|F_{n-k+1}^{(1)}(321,1324)|=n-k$. 2. (ii)

Only one letter is in $\{k+1,\cdots,n\}$ to the left of $k-1$. We claim the letter must be $n$. If not, suppose $x$ is to the left of $k-1$ with $k+1\leq x<n$. Thus $1,x,k-1,n$ is a copy of 1324, which $\pi$ avoids. Therefore $k+1,\cdots,n-1$ appearing after $n$ must be in increasing order and $n$ can be inserted into all the sites between 1 and $k-1$ as follows:

[TABLE]

The number of permutations in this subcase is $k-2$. 3. (iii)

At least two letters in $\{k+1,\cdots,n\}$ are to the left of $k-1$. Suppose that

[TABLE]

We claim that $k+1,\cdots,n$ are all to the left of $k-1$ in increasing order and they are inserted into the same site between 1 and $k-1$.

First, the letters $k+1,\cdots,n$ to the left of $k-1$ are increasing to avoid 321.

Second, we shall prove $k+1,\cdots,n$ are all to the left of $k-1$. If one or more letters of them are to the right of $k-1$, then they must be smaller than those are to the left of $k-1$. Otherwise, we select $x$ from the left of $k-1$ and $y$ from the right of $k-1$ respectively with $x<y$. Thus $1,x,k-1,y$ is a copy of 1324, which $\pi$ avoids. Based on this, we let the letter $s$ be the largest one to the right of $k-1$. The letters in $\{k+1,\cdots,n\}$ to the left of $k-1$ are all greater than $s$. In this case, there are at least two letters to the left of $k-1$, so $s\leq n-2$. Then $s+1$ and the letter immediately to its right together with $s$ would be a forbidden copy of $f$. Therefore there are no letters to the right of $k-1$.

Third, the letters in $\{k+1,\cdots,n\}$ to the left of $k-1$ must be inserted into the same site. Take two letters from $k+1,\cdots,n$ with $x<y$, insert $x$ into the $i$-th site and $y$ into the $j$-th site$(i<j)$, then $i,x,j,y$ would be a copy of 1324.

In this subcase, $k+1,\cdots,n$ can be inserted into all the $k-2$ sites that the left of $k-1$ in increasing order. The number of permutations is $k-2$.

Consequently, the number of permutations in such a case is

[TABLE]

Therefore, for $n\geq 6$,

[TABLE]

∎

Proof of Theorem 2.7.

For $n\geq 3$, we have

[TABLE]

∎

3 Avoiding 321 and two classical patterns of size 4

3.1 Enumerating $F_{n}(321,1423,2143)$

The main result of this subsection is as follows.

Theorem 3.1**.**

For $n\geq 0$, $|F_{n}(321,1423,2143)|=\binom{n}{2}+1$.

Proposition 3.2**.**

For $n\geq 2$,

[TABLE]

Proof.

We can check directly for $n\leq 4$. Let $\pi_{1}=k$ with $2\leq k\leq n$, we shall discuss according to the value of $k$ for $n\geq 5$.

1. (1)

If $k=2$, then avoiding 2143 implies $3,4,\cdots,n$ to the right of $2\ 1$ must be in increasing order. So $\pi=2\ 1\ 3\cdots n$. 2. (2)

If $k=3$, the letters $4,\cdots,n$ must be in increasing order to the right of $3\ 1$ to avoid 2143. Suppose

[TABLE]

and insert 2 into $\pi^{\prime}$. We find that all of the sites labeled from 1 to $n-2$ are available to insert 2. So in this case, the number of permutations is $n-2$. 3. (3)

If $4\leq k\leq n-1$, then both $2,\cdots,k-1$ and $k+1,\cdots,n$ are in increasing order to avoid 321 and 2143 respectively. Assume that

[TABLE]

and we just need to insert $k+1,\cdots,n$ into these sites in increasing order. We find $k+1$ can not be inserted into the site to the left of $k-2$. If $k+1$ is inserted into site labeled from 1 to $k-3$, the letters $1,k+1,k-2,k-1$ would be a copy of 1423, which $\pi$ avoids. Hence $k+1$ can only be inserted into the sites to the right of $k-2$.

1. (i)

If $k+1$ is inserted into the $(k-2)$-th site, we assume that there are $j$ letters between $k-2$ and $k-1$ with $1\leq j\leq n-k$ and there are $n-k-j$ letters remained to be inserted into the $(k-1)$-th site in increasing order. So

[TABLE]

For a fixed $k$, there is only one permutation in terms of the value of $j$. Since $j$ ranges from 1 to $n-k$, the number of permutations in this subcase is $n-k$. 2. (ii)

If $k+1$ is inserted into the $(k-1)$-th site, then

[TABLE]

So in this case, the number of permutations is $n-k+1$. 4. (4)

If $k=n$, due to 321-avoiding condition, the letters $2,\cdots,n-1$ must be in increasing order. Thus $\pi=n\ 1\ 2\cdots n-1$.

For $n\geq 5$, we have

[TABLE]

∎

Proposition 3.3**.**

For $n\geq 2$, $|F_{n}^{(1)}(321,1423,2143)|=n-1.$

Proof.

It is trivial to prove for $n\leq 3$. Let $\pi_{2}=k.$ We claim $k=2$ or $k=3$ when $n\geq 4$. If not, assume that $k\geq 4$. If 2 is to the left of 3, then $1,k,2,3$ is a copy of 1423. If 3 is to the left of 2, the letters $k,3,2$ is a copy of 321. So $k=2$ or $k=3$.

• •

$k=3$. The Fishburn condition forces $\pi_{3}=2$. Moreover, to avoid 2143, the letters $4,\cdots,n$ appearing after $3\ 2$ must be in increasing order. So

[TABLE]

• •

$k=2$. We assume $\pi=1\ 2\cdots j\ x\cdots$ with $2\leq j\leq n-2$ and $x\geq j+2$. Notice an exceptional case is $\pi=1\ 2\cdots n$. We claim $x=j+2$. We shall deduce contradiction by discussing the order of $j+1$ and $j+2$ in $\pi$. If $j+1$ is to the left of $j+2$, then $j,x,j+1,j+2$ is a copy of 1423. Whereas if $j+2$ is to the left of $j+1$, then $x,j+2,j+1$ is a copy of 321, a contradiction. So $x=j+2$. Furthermore, $j+1$ must be immediately to the right of $j+2$. If not, $j+2$ and the letter immediately to its right together with $j+1$ would be a copy of $f$. Besides, to avoid 2143, the letters $j+3\cdots,n$ appearing after $j+2\ j+1$ must be in increasing order. So

[TABLE]

Together with the exceptional permutation $\pi=1\ 2\cdots n$, the number of permutations in this case is $(n-3)+1=n-2$, in which $n-3$ is due to $j$ ranging from 2 to $n-2$.

Therefore, for $n\geq 4$,

[TABLE]

∎

Proof of Theorem 3.1.

We can check directly for $n=0$ and $n=1$. For $n\geq 2$, we get

[TABLE]

∎

3.2 Enumerating $F_{n}(321,3142,2143)$

The main result of this subsection is as follows.

Theorem 3.4**.**

For $n\geq 0$, $|F_{n}(321,3142,2143)|=\binom{n}{2}+1$.

Proposition 3.5**.**

For $n\geq 2$, $|F_{n}^{(2)}(321,3142,2143)|=n-1.$

Proof.

It is easy to show that the proposition holds for $n\leq 3$. Now we consider $n\geq 4$. Let $\pi_{1}=k$ with $2\leq k\leq n$.

1. (1)

If $k=2$, avoiding 2143 forces the letters $3,4,\cdots,n$ to the right of $2\ 1$ must be in increasing order. So $\pi=2\ 1\ 3\cdots n$. 2. (2)

$3\leq k\leq n-1$. First, to avoid 321, the letters $2,\cdots,k-1$ appearing after $k$ must be in increasing order. Second, to avoid 2143, the letters $k+1,\cdots,n$ appearing after $k$ and 1 must be in increasing order. Third, to avoid 3142, the letter $k+1$ must be to the right of $k-1$. If $k+1$ appears to the left of $k-1$, then $k,1,k+1,k-1$ would be a copy of 3142, which $\pi$ avoids. So

[TABLE] 3. (3)

If $k=n$, then to avoid 321, the letters $2,\cdots,n-1$ must be in increasing order. So $\pi=n\ 1\ 2\cdots n-1$.

Fixing $k$, we can determine a unique permutation. The number of permutations, which depends on $k$ ranging from 2 to $n$, is $n-1$.

Therefore, for $n\geq 2$, we have

[TABLE]

∎

Proposition 3.6**.**

For $n\geq 2$,

[TABLE]

Proof.

It is easy to prove this proposition holds for $n\leq 4$. For $n\geq 5$, if $\pi$ has no descents, we get $\pi=1\ 2\cdots n$. For $\pi$ with at least one descent, we assume

[TABLE]

where $1\leq k\leq n-2$ and $k+2\leq x\leq n$. Under our assumption, there is at least one letter after $x$. Next we will divide our proof into several cases according to the value of $x$.

1. (1)

$x=n$. To avoid 321, the letters $k+1,\cdots,n-1$ appearing after $n$ must be in increasing order. So $\pi=1\ 2\cdots k\ n\ k+1\cdots n-1.$ 2. (2)

$k+3\leq x\leq n-1$. First, to avoid 321, the letters $k+1,\cdots,x-1$ appearing after $x$ must be in increasing order. Assume that

[TABLE]

We need to insert $x+1,\cdots,n$ into $\pi^{\prime}$ to get $\pi$. We noticed that the Fishburn condition forces that no letters in $\{x+1,\cdots,n\}$ can be inserted into the first site. So $k+1$ is the letter immediately to the right of $x$. Second, to avoid 2143, the letters $x+1,\cdots,n$ appearing after $x$ and $k+1$ must be in increasing order. Furthermore, we claim that $x+1$ must be to the right of $x-1$. If $x+1$ is inserted into sites labeled from 2 to $x-k-1$, then $x,k+1,x+1,x-1$ would be a copy of 3142. Actually, $x+1$ must be to the right of $x-1$ in increasing order and

[TABLE] 3. (3)

$x=k+2$. Let $y$ be the letter immediately to the right of $k+2$. We claim $y=k+1$. If $y\neq k+1$, then $y$ must be greater than $k+2$. Clearly $k+2,y,k+1$ is a copy of $f$, which $\pi$ avoids. So $y=k+1$. Furthermore, to avoid 2143, the letters $k+3,\cdots,n$ appearing after $k+2$ and $k+1$ must be in increasing order. Therefore, $\pi=1\cdots k\ k+2\ k+1\ k+3\cdots n.$

Fixing $k$ and $x$, we can determine a unique permutation. Since $x$ ranges from $k+2$ to $n$ and $k$ ranges from 1 to $n-2$, the number of permutations is

[TABLE]

Together with the exceptional case $\pi=1\ 2\cdots n$, we get

[TABLE]

∎

Proof of Theorem 3.4.

The cases when $n<2$ are easy to check. For $n\geq 2$, we have

[TABLE]

∎

3.3 Enumerating $F_{n}(321,2143,3124)$

The main result of this subsection is as follows.

Theorem 3.7**.**

For $n\geq 0$, $|F_{n}(321,2143,3124)|=\binom{n}{2}+1$.

Proof.

The enumeration of $F_{n}(321,2143,3124)$ follows similar lines as that of $F_{n}(321,3142,2143)$, except for two cases. However, they do not influence the total enumeration. We now consider the two different cases of $F_{n}(321,2143,3124)$ from that of $F_{n}(321,3142,2143)$.

One different case is Case (2) in the proof of Proposition 3.5. Suppose that $3\leq k\leq n-1$. In order to avoid 321 and 2143, we have

[TABLE]

and we consider how to insert $k+1,\cdots,n$ into $\pi^{\prime}$ in increasing order. We claim that $k+1,\cdots,n$ should be inserted into the first site. If not, $k,1,2,k+1$ would be a copy of 3124, which $\pi$ avoids. So the form of $\pi$ in this case is as follows,

[TABLE]

The other one is Case (2) in the proof of Proposition 3.6. Suppose that $k+3\leq x\leq n-1$. Similarly, we need to insert $x+1,\cdots,n$ into the sites as labeled below

[TABLE]

If we insert $x+1$ into the sites except the first, then $x,k+1,k+2,x+1$ would be a copy of 3124. So $x+1,\cdots,n$ must be inserted into the first site, and

[TABLE]

Therefore,

[TABLE]

∎

3.4 Enumerating $F_{n}(321,2143,4123)$

The main result of this subsection is as follows.

Theorem 3.8**.**

For $n\geq 0$, $|F_{n}(321,2143,4123)|=\binom{n}{2}+1$.

Proposition 3.9**.**

For $n\geq 2$, $|F_{n}^{(2)}(321,2143,4123)|=n-1$.

Proof.

It is trivial to show the proposition hold for $n\leq 3$. Consider $n\geq 4$. Let $\pi_{1}=k$. We shall prove $k<4$. If $k\geq 4$, there exist $k-1$ and $k-2$ to the right of 1. Avoid 321 implies $k-2$ and $k-1$ are in increasing order, whereas avoiding 4123 implies they are in decreasing order, a contradiction. So either $k=2$ or $k=3$.

1. (1)

If $k=2$, to avoid 2143, the letters $3,4,\cdots,n$ must be in increasing order. So $\pi=2\ 1\ 3\cdots n$. 2. (2)

If $k=3$, the forbidden pattern 2143 means $4,\cdots,n$ are in increasing order. Let

[TABLE]

and we should insert 2 into $\pi^{\prime}$ now. It is easy to check that 2 can be inserted into all the $n-2$ sites, so the number of permutations in this case is $n-2$.

Therefore, for $n\geq 4$,

[TABLE]

∎

Proposition 3.10**.**

For $n\geq 2$,

[TABLE]

Proof.

We can directly check this proposition is true for $n\leq 3$. If $n\geq 4$, we assume that $\pi$ has the following form $\pi=1\ 2\cdots k\cdots$. Take $\pi=1\ 2\cdots n$ as an exceptional case. Next we consider $\pi=1\cdots k\ x\cdots$ with $1\leq k\leq n-2$ and $x\geq k+2$. We claim either $x=k+2$ or $x=k+3$. If the claim would not hold, then $x\geq k+4$. We discuss the order of $k+1$, $k+2$ and $k+3$ to deduce a contradiction. To avoid 4123, $x,k+1,k+2,k+3$ can not be in increasing order. However, if $k+1,k+2,k+3$ contain at least one descent, then $x$ and the decreasing pair would be a copy of 321, a contradiction. So either $x=k+2$ or $x=k+3$.

1. (1)

$x=k+2$. Avoiding the Fishburn pattern $f$ forces $k+1$ must be immediately to the right of $k+2$. Moreover, to avoid 2143, the letters $k+3,\cdots,n$ appearing after $k+2$ and $k+1$ must be in increasing order. So $\pi=1\ 2\cdots k\ k+2\ k+1\ k+3\cdots n.$ 2. (2)

$x=k+3$. We claim $k+1$ must be immediately to the right of $k+3$. If not, let $y$ be the letter immediately to the right of $k+3$ instead of $k+1$, namely $y=k+2$ or $y\geq k+4$. If $y=k+2$, then $k+3,k+2,k+1$ would be a copy of 321. If $y\geq k+4$, then $k+3,y,k+2$ would be a copy of $f$. So $y=k+1$.

Next we consider the relation of $n$ and $k+3$. If $n=k+3$, the letter $k+2$ is to the right of $k+1$ and we have determined $\pi=1\ 2\cdots k\ k+3\ k+1\ k+2$. If $n>k+3$, avoiding 2143 implies the letters $k+4,\cdots,n$ appearing after $k+3$ and $k+1$ must be in increasing order. Let

[TABLE]

and we insert $k+2$ into it. We find all the $n-k-2$ sites are available to insert $k+2$. So the number of permutations in this case is $n-k-2$.

For a fixed $k$, the number of permutations which depends on the value of $k$ is $1+(n-k-2)=n-k-1.$ Together with the exceptional case $\pi=1\ 2\cdots n$, we obtain

[TABLE]

∎

Proof of Theorem 3.8.

We can easily prove the cases when $n<2$. For $n\geq 2$, we get

[TABLE]

∎

3.5 Enumerating $F_{n}(321,1423,3124)$

In this subsection and the following two subsections, the enumeration involves the famous Fibonacci numbers, see A000045 in OEIS [9]. The Fibonacci numbers, denoted by $F_{n}$, satisfy initial conditions $F_{0}=F_{1}=1$ and recurrence relation $F_{n}=F_{n-1}+F_{n-2}$ for $n\geq 2$.

It is worth mentioning that, in the enumeration involving Fibonacci numbers, we consider the first $k$ letters of $\pi$ of some kind of classifications, which are also the rearrangement of $1\ 2\cdots\ k$, as a whole part to play the role of 1. In this way, we can reduce the enumeration to those permutations of lower length, which can be turned into the enumeration we have proved before, to facilitate enumeration and induction.

The main result of this subsection is as follows.

Theorem 3.11**.**

For $n\geq 4$,

[TABLE]

Proposition 3.12**.**

For $n\geq 2$, we get

[TABLE]

Proof.

We shall prove by induction on $n$. For $n=2$ we get

[TABLE]

Suppose this equality holds for positive integers less than $n$. We shall prove it holds for $n$ as well. We claim $\pi_{2}=2$ or $\pi_{2}=3$. If not, we have $\pi_{2}\geq 4$ and 2,3 must appear to the right of $\pi_{2}$. If 2 is to the left of 3, then $1,\pi_{2},2,3$ is a copy of 1423. If 3 is to the left of 2, then $\pi_{2},3,2$ is a copy of 321. Therefore, $\pi_{2}<4$. Next we classify $F_{n}^{(1)}(321,1423,3124)$ into three cases.

1. (1)

$\pi=1\ 2\cdots n$. 2. (2)

$\pi=1\ 2\cdots k\ x\ y\cdots$ with $2\leq k\leq n-2$ and $x\geq k+2$. We claim $x=k+2$ and $y=k+1$. First, we prove $x=k+2$. If $x>k+2$, we consider the order of $k+1$ and $k+2$. Avoiding 321 implies $k+1$ is to the left of $k+2$. But at this time, the letters $k,x,k+1,k+2$ would be a copy of 1423. So $x=k+2$. Second, we prove $y=k+1$. If $y\neq k+1$, then $k+2,y,k+1$ would be a copy of $f$. Now we have shown that the first $k+2$ letters of $\pi$ are $1,2,\cdots,k,k+2,k+1$. We can check that any pair or triple from them can not be parts of the copy of 321,1423,3124 and $f$. The first $k+2$ letters, as a whole, can play the role of 1 as the minimum letter in any copy of 321,1423,3124 and $f$. So, by the induction hypothesis, the number of permutations in this case is

[TABLE] 3. (3)

$\pi=1\ 3\ x\cdots$. The Fishburn condition forces $x=2$. Similar to the above case, we can treat 1,3,2 as one letter to play the role of 1. The number of permutations in this case is

[TABLE]

Therefore,

[TABLE]

where the third equality follows from a property of Fibonacci numbers. For more properties of Fibonacci numbers, see [8, Chapter 1]. ∎

Proposition 3.13**.**

For $n\geq 4$,

[TABLE]

Proof.

Suppose $\pi_{1}=k.$ We claim the value of $k$ can be chosen from $\{2,3,n\}$. If not, we assume that $4\leq k\leq n-1$. There are at least two letters less than $k$ to the right of $k\ 1$ and they can not contain descents to avoid 321. Next suppose

[TABLE]

and consider which site after 1 we can insert $n$ into. If $n$ is inserted into the first site, then $1,n,2,k-1$ would be a copy of 1423. Whereas if $n$ is inserted into those sites to the right of 2, then $k,1,2,n$ would be a copy of 3124, which is a contradiction. So either $k<4$ or $k=n$.

1. (1)

$k=2$. The letters 2,1 can be as one letter to play the role of 1. The number of permutations in this case is

[TABLE] 2. (2)

$k=3$. First we have $\pi_{n}=2$. If there is a letter $x\geq 4$ to the right of 2, then $3,1,2,x$ is a copy of 3124. Second, avoiding 321 forces $4,\cdots,n$ to the left of 2 must be in increasing order. So

[TABLE] 3. (3)

$k=n$. To avoid 321, the letters $2,\cdots,n-1$ must be in increasing order. So

[TABLE]

Thus,

[TABLE]

∎

Proof of Theorem 3.11.

For $n\geq 4$, we have

[TABLE]

∎

3.6 Enumerating $F_{n}(321,1423,4123)$

The main result of this subsection is as follows.

Theorem 3.14**.**

For $n\geq 1$,

[TABLE]

Proposition 3.15**.**

For $n\geq 4$,

[TABLE]

Proof.

We first claim $\pi_{2}=2$ or $\pi_{2}=3$. We shall prove by contradiction. If $\pi_{2}\geq 4$, then 2 and 3 must be to the right of $\pi_{2}$. To avoid 321, they are in increasing order. However, the letters $1,\pi_{2},2,3$ would be a copy of 1423, a contradiction.

Next we shall prove the equality (1) by induction on $n$. For $n=4$ we have

[TABLE]

So,

[TABLE]

Suppose equality (1) holds for positive integers less than $n$. We shall prove it holds for $n$ as well. We classify $F_{n}^{(1)}(321,1423,4123)$ into two cases according to the value of $\pi_{2}$.

1. (1)

If $\pi_{2}=2$, then 1,2 can be treated as a whole part to play the role of 1. So the number of permutations in this case can be reduced to

[TABLE] 2. (2)

If $\pi_{2}=3$, then avoiding $f$ indicates $\pi_{3}=2$. Thus the first three letters 1,3,2 can be considered as one letter to play the role of 1. The number of permutations in this case is

[TABLE]

Therefore,

[TABLE]

∎

Proposition 3.16**.**

For $n\geq 4$,

[TABLE]

Proof.

We claim $\pi_{1}=2$ or $\pi_{1}=3$. If not, suppose that $\pi_{1}\geq 4$. The letters 2,3 are to the right of 1. If they are increasing order, then $\pi_{1},1,2,3$ would be a copy of 4123. Whereas if they are in decreasing order, then $\pi_{1},3,2$ would be a copy of 321. Next we classify $F_{n}^{(2)}(321,1423,4123)$ into two cases according to the value of $\pi_{1}$.

1. (1)

If $\pi_{1}=2$, the letters 2,1 can be seen as one letter to play the role of 1. The number of permutations in this case is

[TABLE] 2. (2)

If $\pi_{1}=3$, we consider the position of 2.

1. (i)

$\pi_{3}=2$. We can treat three letters 3,1,2 as one letter to play the role of 1. By equality (1), we obtain that the number of permutations in this case is

[TABLE] 2. (ii)

$\pi_{3}\neq 2$. We claim

[TABLE]

with $4\leq k\leq n$. First we shall prove $\pi_{3}=4$ by contradiction. We assume $\pi_{3}>4$ and discuss the value of $\pi_{4}$. On one hand, if $\pi_{4}=2$, the letter 4 is to the right of 2 and $1,\pi_{3},2,4$ would be a copy of 1423. On the other hand, if $\pi_{4}\neq 2$, then to avoid 321, $\pi_{3}$ and $\pi_{4}$ must be in increasing order since 2 is to the right of them. Under the assumption of $\pi_{3}>4$, we have $\pi_{3}-1$ must be to the right of $\pi_{4}$ and $\pi_{3},\pi_{4},\pi_{3}-1$ would be a copy of $f$, which is a contradiction.

Second, suppose that there are $\ell$ letters between 1 and 2. If $\ell=1$, then the letter is 4. If $\ell>1$, we denote them by $a_{1},a_{2},\cdots,a_{\ell}$ with $a_{1}=4$. Next we show that $a_{i+1}=a_{i}+1$ for $1\leq i\leq\ell-1$. The letters $a_{1},a_{2},\cdots,a_{\ell}$ to the left of 2 must be increasing to avoid 321. Furthermore, if there exists $j\in[\ell]$ such that $a_{j}\neq a_{j-1}+1$, then $a_{j}-1$ is to the right of 2. Thus $1,a_{j},2,a_{j}-1$ is a copy of 1423. Let $a_{\ell}=k$ and substitute $a_{1},a_{2},\cdots,a_{\ell}$ with $4,5,\cdots,k$, where $k$ ranges from 4 to $n$.

The first $k$ letters $3,1,4,\cdots,k,2$ can be seen as one letter to play the role of 1. So

[TABLE]

Therefore,

[TABLE]

∎

Proof of Theorem 3.14.

We can check directly for $n\leq 3$. For $n\geq 4$, we have

[TABLE]

∎

3.7 Enumerating $F_{n}(321,3124,4123)$

The main result of this subsection is as follows.

Theorem 3.17**.**

For $n\geq 1$,

[TABLE]

Proposition 3.18**.**

For $n\geq 4$,

[TABLE]

Proof.

We claim $\pi_{2}\leq 4$. If $\pi_{2}\geq 5$, the letters 2,3 and 4 are to the right of $\pi_{2}$. The condition avoiding 321 implies 2,3 and 4 are in increasing order. But $\pi_{2},2,3,4$ would be a copy of 4123, which $\pi$ avoids. So $\pi_{2}\leq 4$.

We shall prove (2) by induction. For $n=4$ we have

[TABLE]

So,

[TABLE]

Suppose this equality holds for positive integers less than $n$, and we prove it holds for $n$ as well. We divide $F_{n}^{(1)}(321,3124,4123)$ into three cases according to the value of $\pi_{2}$.

1. (1)

$\pi_{2}=2$. We can treat 1,2 as a whole to play the role of 1. So by the induction hypothesis, the number of permutations in this case can be reduced to

[TABLE] 2. (2)

$\pi_{2}=3$. Then avoiding $f$ forces $\pi_{3}=2$. The first three letters 1,3,2 can be seen the smallest letter. Thus the number of permutations in this case is

[TABLE] 3. (3)

$\pi_{2}=4$. We have

[TABLE]

First we prove $\pi_{3}=2$ by contradiction. Suppose that $\pi_{3}\geq 3$. On one hand, if $\pi_{3}=3$, the letters 4,3,2 create a copy of 321. On the other hand, if $\pi_{3}>3$, actually $\pi_{3}\geq 5$, then $4,\pi_{3},3$ would be a copy of $f$. So $\pi_{3}=2$. Second, we have $\pi_{n}=3$. If not, $4,2,3,\pi_{n}$ is a copy of 3124 because 3 appears after 2 and before $\pi_{n}$. Now it just leaves the sites of $5,\cdots,n$ to the left of 3 undetermined for $n\geq 5$. To avoid 321, they must be in increasing order.

Thus,

[TABLE]

∎

Proposition 3.19**.**

For $n\geq 4$,

[TABLE]

Proof.

We claim $2\leq\pi_{1}\leq 3$. If not, suppose $\pi_{1}\geq 4$. If 2,3 are increasing, then $\pi_{1},1,2,3$ is a copy of 4123. Whereas if they are in decreasing order, then $\pi_{1},3,2$ is a copy of 321. So $\pi_{1}=2$ or $\pi_{1}=3$. Thus the number of $F_{n}^{(2)}(321,3124,4123)$ can be classified into two cases according to the value of $\pi_{1}$.

If $\pi_{1}=2$, we can treat 2,1 as one letter to play the role of 1. The number of permutations in this case can be reduced to

[TABLE]

If $\pi_{1}=3$, we have $\pi=3\ 1\ 4\cdots n\ 2.$ First we shall show $\pi_{n}=2$. If $\pi_{n}\neq 2$, then $3,1,2,\pi_{n}$ would be a copy of 3124. Second, to avoid 321, the letters $4,\cdots,n$ appearing before 2 must be in increasing order.

Therefore,

[TABLE]

∎

Proof of Theorem 3.17.

We can check directly for $n\leq 3$. For $n\geq 4$, we have

[TABLE]

∎

4 Avoiding 321 and a classical pattern of size 5

4.1 Enumerating $F_{n}(321,14253)$

The main result of this subsection is as follows.

Theorem 4.1**.**

For $n\geq 1$, we have

[TABLE]

Recall that the direct sum of two permutations $\sigma$ and $\tau$ of length $k$ and $\ell$ respectively, denoted by $\sigma\oplus\tau$, is a permutation of length $k+\ell$ consisting of $\sigma$ followed by $\tau^{\prime}$ (every letter in $\tau$ plus $k$). For example, $12\oplus 123=12345$.

In order to prove Theorem 4.1, we first enumerate $F_{n}(321,3142)$.

Lemma 4.2**.**

For $n\geq 1$, we have

[TABLE]

Proof.

We shall prove this lemma by induction. One can easily check that the equality (4) holds for $n\leq 3$. Suppose the equality (4) holds for positive integers less than $n$ with $n\geq 4$. Next we shall prove it holds for $n$ as well. We classify $F_{n}(321,3142)$ into several cases according to the first two letters of permutations in $F_{n}(321,3142)$.

1. (1)

$\pi=1\ 2\oplus\tau$. Clearly $\tau$ is in $F_{n-2}(321,3142)$. Thus by the induction hypothesis, the number of permutation in this case is $|F_{n-2}(321,3142)|=2^{n-3}$. 2. (2)

$\pi=1\ m\cdots$ with $3\leq m\leq n-1$. We claim the first $m$ letters of $\pi$ are $1,m,2,\cdots,m-1$. First, the sequence $2,\cdots,m-1$ appearing after $m$ must be increasing to avoid 321. Suppose

[TABLE]

and we should insert $m+1,\cdots,n$ into $\pi^{\prime}$ to get $\pi$. Second, the letter $x\in\{m+1,\cdots,n\}$ can not be inserted into the first site since $m,x,m-1$ would be a copy of $f$. Third, $x$ can not be inserted into sites labeled from 2 to $m-2$ $m,2,x,m-1$ would be a copy of 3142. It follows that $m+1,\cdots,n$ can only be to the right of $m-1$. Therefore, $\pi=1\ m\ 2\cdots m-1\oplus\tau$, where $\tau\in F_{n-m}(321,3142)$. The number of permutations in this case is $|F_{n-m}(321,3142)|=2^{n-m-1}$. 3. (3)

$\pi=1\ n\cdots$. Then to avoid 321, the letters $2,\cdots,n-1$ appearing after $n$ must be in increasing order. Thus $\pi=1\ n\ 2\cdots n-1$. 4. (4)

$\pi=2\ 1\oplus\tau$ with $\tau\in F_{n-2}(321,3142)$. The number of permutations in this case is $|F_{n-2}(321,3142)|=2^{n-3}$. 5. (5)

$\pi=k\ 1\cdots$ with $3\leq k\leq n-1$. We claim

[TABLE]

where $\tau\in F_{n-k}(321,3142)$. To avoid 321, the letters $2,\cdots,k-1$ appearing after $k$ must be in increasing order. Now we suppose

[TABLE]

We next insert $k+1,\cdots,n$ into $\pi^{\prime}$ to obtain $\pi$. To avoid 3142, the letter $x\in\{k+1,\cdots,n\}$ can not be inserted into the sites labeled from 1 to $k-2$ since $k,1,x,k-1$ would be a copy of 3142. Indeed, $k+1,\cdots,n$ can only be inserted to the right of $k-1$. Therefore, in this case, the number of permutations is $|F_{n-k}(321,3142)|=2^{n-k-1}$. 6. (6)

$\pi=n\ 1\cdots$. Avoiding 321 implies the letters $2,\cdots,n-2$ appearing after $n$ must be in increasing order. So $\pi=n\ 1\ 2\cdots n-1$.

Therefore, for $n\geq 1$, we have

[TABLE]

∎

Proof of Theorem 4.1.

We can directly check that Theorem 4.1 holds for $n\leq 5$. For $n\geq 6$, we will classify according to the site where 1 appears.

Let $\pi$ be a permutation in $F_{n}^{(1)}(321,14253)$. If $\pi_{1}=1$, then $\pi=1\oplus\tau$ with $\tau\in F_{n-1}(321,3142)$. By equality (4), the number of permutations is $|F_{n-1}(321,3142)|=2^{n-2}$.

For $\pi_{2}=1$, we proceed to classify $F_{n}^{(2)}(321,14253)$ according to the value of $\pi_{1}$.

1. (1)

$\pi_{1}=2$. Then $\pi=2\ 1\oplus\tau$ with $\tau\in F_{n-2}(321,3142)$. Hence, the number of permutations in this case is $|F_{n-2}(321,3142)|=2^{n-3}$. 2. (2)

$\pi_{1}=k$ with $3\leq k\leq n-3$. First, the letters $2,\cdots,k-1$ appearing after $k$ must be in increasing order. Let $x$ and $y$ be two letters chosen from $\{k+1,\cdots,n\}$. Second, to avoid 321, the letters $x,y$ appearing before $k-1$ must be in increasing order. If $k=3$, there is only one space between 1 and 2. If $k\geq 4$, there are at least two spaces between 1 and $k-1$. Suppose

[TABLE]

We claim the letters $k+1,\cdots,n$ can be inserted into only one site labeled from 1 to $k-2$ in increasing order. Otherwise, suppose there are two letters from $k+1,\cdots,n$ inserted into two distinct sites between 1 and $k-1$, say $x$ is to the left of $i$ and $y$ is to the right of $i$ with $2\leq i\leq k-2$. Then $1,x,i,y,k-1$ would be a copy of 14253.

We next insert $k+1,\cdots,n$ into $\pi^{\prime}$ to get $\pi$. There are two subcases to discuss.

1. (i)

All the letters $k+1,\cdots,n$ are inserted into the $(k-1)$-th site. Thus, the first $k$ letters of $\pi$ are $k,1,2\cdots,k-1$. Therefore, $\pi=k\ 1\ 2\cdots k-1\oplus\tau$ with $\tau\in F_{n-k}(321,3142)$. So the number of permutations in this case is $|F_{n-k}(321,3142)|=2^{n-k-1}$. 2. (ii)

The letters $k+1,\cdots,n$ are inserted in to two sites: the $i$-th and the $(k-1)$-th, where $1\leq i\leq k-2$. There are $2^{n-k}-1$ ways to divide $k+1,\cdots,n$ into two sites. As $i$ has $k-2$ choices, the number of permutations in this subcase is $(k-2)(2^{n-k}-1)$. 3. (3)

$\pi_{1}=n-2$. Then $2,\cdots,n-3$ appearing after $n-2$ must be in increasing order. Suppose

[TABLE]

where $1\leq i\leq n-3$. We can insert $n-1$ and $n$ into $\pi^{\prime}$ in the following ways.

• •

$n-1$ is in the first site and $n$ is in the second site.

• •

$n$ is in the first site and $n-1$ is in the second site.

• •

$n-1$ and $n$ are both in the first site in increasing order.

• •

$n-1$ and $n$ are both in the second site.

The first three cases are related to $1\leq i\leq n-4$. In the last case, $n-1$ and $n$ can be inserted in both increasing and decreasing order. Thus the number of permutations when $\pi_{1}=n-2$ is $3(n-4)+2=3n-10$. 4. (4)

$\pi_{1}=n-1$. Then $2,\cdots,n-2$ appearing after $n-1$ must be in increasing order. Suppose $\pi^{\prime}=n-1\ 1\ ^{1}\ 2\ ^{2}\cdots\ ^{n-3}\ n-2\ ^{n-2}$ and we should insert $n$ into $\pi^{\prime}$ to get $\pi$. Actually, all the $k-2$ sites are available to insert $n$. Thus the number of permutations in this case is $n-2$. 5. (5)

$\pi_{1}=n$. Then to avoid 321, the letters $2,\cdots,n-1$ appearing after $n$ must be in increasing order. So $\pi=n\ 1\ 2\cdots n-1$.

Therefore, for $n\geq 6$,

[TABLE]

∎

4.2 Enumerating $F_{n}(321,21354)$

The main result of this subsection is as follows.

Theorem 4.3**.**

For $n\geq 1$, we have

[TABLE]

Before proving Theorem 4.3, we shall enumerate $F_{n}(321,132)$ first.

Lemma 4.4**.**

For $n\geq 1$, we have

[TABLE]

Proof.

By Lemma 1.1, we make the following classification according to the site letter 1 appears.

1. (1)

$\pi_{1}=1$. To avoid 132, the letters $2,3,\cdots,n$ appearing after 1 must be in increasing order. So in this case, $\pi=1\ 2\cdots n$. 2. (2)

$\pi_{2}=1$. We proceed to classify according to the value of $\pi_{1}$.

• •

$\pi_{1}=2$. The sequence $3,\cdots,n$ to the right of $2\ 1$ must be increasing to avoid 132. Thus $\pi=2\ 1\ 3\cdots n$.

• •

$\pi_{1}=k$ with $3\leq k\leq n$. Then $2,\cdots,k-1$ appearing after $k$ must be increasing to avoid 321. Moreover, to avoid 132, the letters $k+1,\cdots,n$ appearing after 1 must be increasing and they are to the right of $k-1$. So $\pi=k\ 1\ 2\cdots k-1\ k+1\cdots n$.

Therefore,

[TABLE]

∎

For our convenience, we shall denote by $F_{n}^{<k12\cdots\ell>}(321,21354)\ (1\leq\ell\leq k-1)$ the set of the Fishburn permutations of length $n$ that avoid both 321 and 21345, and begin with $k\ 1\ 2\cdots\ell$. Denote by $F_{n}^{<k12\cdots\bar{\ell}>}(321,21354)$ the set of those permutations begin with $k\ 1\ 2\cdots\ell-1$ but not $k\ 1\ 2\cdots\ell$.

Proof of Theorem 4.3.

Our proof is by induction. One can directly check the equality (5) holds for $n\leq 3$. Suppose the equality (5) holds for $n-1$. We shall prove it holds for $n$ as well. Let $\pi\in F_{n}(321,21354)$. If $\pi_{1}=1$, then $\pi=1\oplus\tau$ with $\tau\in F_{n-1}(321,21354)$. Thus it follows from the induction that

[TABLE]

Therefore it suffices to prove the following equality

[TABLE]

For the enumeration of $F_{n}^{(2)}(321,21354)$, our classification is according to the value of $\pi_{1}$.

1. (1)

$\pi_{1}=2$. Then $\pi=2\ 1\oplus\tau$ with $\tau\in F_{n-2}(321,132)$. By Lemma 4.4, the number of permutations in this case is $|F_{n-2}(321,132)|=n-2$. 2. (2)

$\pi_{1}=3$. Next we proceed to consider the site where 2 appears.

1. (i)

Letter 2 is immediately to the right of 1. We have

[TABLE]

where $\tau\in F_{n-3}(321,132)$. The number of permutations in this subcase is

[TABLE] 2. (ii)

There is only one letter $m$ between 1 and 2 with $4\leq m\leq n$.

• •

$m=4$. Then $5,\cdots,n$ appearing after $3\ 1\ 4$ must be increasing to avoid 21354. So $\pi=3\ 1\ 4\ 2\ 5\cdots n$.

• •

$m\geq 5$. First, to avoid 321, the sequence $4,\cdots,m-1$ appearing after $m$ must be increasing. Second, avoiding 21354 implies that the sequence $m+1,\cdots,n$ appearing after $3\ 1\ m$ must be increasing and there are no letters in $m+1,\cdots,n$ can be inserted into the sites between 4 and $m-1$. The former is easy to see. As for the latter, if $x\in\{m+1,\cdots,n\}$ are inserted into one site between 4 and $m-1$, then $3\ 1\ 4\ x\ m-1$ would be a copy of 21354. Hence, we can consider $4\cdots m-1$ as one letter and insert it into the $n-m+1$ sites that the letters $m+1,\cdots,n$ formed. There are $n-m+1$ ways to insert $4\cdots m-1$ to get $\pi$.

Thus, in this subcase, the number of permutations is

[TABLE] 3. (iii)

There are at least two letters between 1 and 2. Then we claim the letters between 1 and 2 are $4,5,\cdots,m$ in increasing order with $5\leq m\leq n$. We shall take $\pi_{3}$ and $\pi_{4}$ for example to illustrate. First, the letters $\pi_{3}$ and $\pi_{4}$ appearing before 2 must be increasing to avoid 321. Second, we have $\pi_{3}=4.$ If not, $\pi_{3}-1$ appears to the right of 2 and $\pi_{3},\pi_{4},\pi_{3}-1$ would be a copy of $f$. Third, avoiding $f$ forces $\pi_{4}=5$ since $3,1,4,\pi_{4},\pi_{4}-1$ would be a copy of 21354 if $\pi_{4}\neq 5$. For the same reason, we can prove $\pi_{5}=6$ and so forth. Thus the first $m$ letters of $\pi$ are $3,1,4,\cdots,m,2$. To avoid 21354, the letters $m+1,\cdots,n$ appearing after $3,1,m$ must be in increasing order. So

[TABLE]

where $5\leq m\leq n$. Therefore, the number of permutations in this subcase is $n-4$.

It is worth mentioning that the number of those permutations with $\pi_{1}=3$ and $\pi_{2}=1$ but $\pi_{3}\neq 2$ is

[TABLE]

Thus, we have

[TABLE] 3. (3)

$\pi_{1}=k$ with $4\leq k\leq n-1$. The number of permutations in this case is

[TABLE]

For permutation $\pi$ begins with $k\ 1\ 2\cdots k-1$, we have $\pi=k\ 1\ 2\cdots k-1\oplus\tau$ with $\tau\in F_{n-k}(321,132)$. Thus

[TABLE]

In the following part, we discuss the enumeration of $F_{n}^{<k1\bar{2}>}(321,21354)$ for $k\geq 4$. Other enumerations $F_{n}^{<k12\cdots\bar{\ell}>}(321,21354)$ for $3\leq\ell\leq k-1$ can be reduced to this form. We classify according to the number of letters between 1 and 2.

1. (i)

There is only one letter $m$ between 1 and 2. Suppose $\pi=k\ 1\ m\ 2\cdots$. Then to avoid 321, we have $m>k$ because $k$ appears before $m\ 2$. Here we consider $m\geq k+2$. The case of $m=k-1$ is discussed later in (ii). Next we need to consider the arrangement of the remained letters $\{3,\cdots,k-1\}\cup\{k+1,\cdots,m-1\}\cup\{m+1,\cdots,n\}$. First, the sequence $3,\cdots,k-1$, the sequence $k+1,\cdots,m-1$, and the sequence $m+1,\cdots,n$ must be increasing to avoid the pattern 321, 321, 21354 respectively. Second, the sequence $k+1,\cdots,m-1$ must be to the right of the increasing sequence $3\cdots k-1$ to avoid 321. Third, there are no letters in $m+1,\cdots,n$ can be inserted into the sites between $k+1$ and $m-1$. If $x\in\{m+1,\cdots,n\}$ is inserted into one site between $k+1$ and $m-1$, then $k,1,k+1,x,m-1$ would be a copy of 21354. Hence, we can consider $k+1,\cdots,m-1$ as one letter and thus consider $\{3,\cdots,k-1\}\cup\{k+1,\cdots,m-1\}\cup\{m+1,\cdots,n\}$ as two increasing sequences, one is of length $k-2$ consisting of letters in the first two sets, the other is of length $n-m$ consisting of $\{m+1,\cdots,n\}$. Moreover, we can consider its counting as the number of permutations of the multiset $\{(k-2)\cdot A,(n-m)\cdot B\}$, which is $\binom{n-m+k-2}{k-2}$. 2. (ii)

There are at least two letters between 1 and 2. Then we claim the letters between 1 and 2 are consecutive letters from $k+1$ to $m$ in increasing order with $k+1\leq m\leq n$(notice that $m=k+1$ was discussed here). First, the letters between 1 and 2 must be greater than $k$ and they are increasing to avoid 321. Second, if $\pi_{3}\neq k+1$, the letters $\pi_{3},\pi_{4},\pi_{3}-1$ would be a copy of $f$. So $\pi_{3}=k+1$. Third, we have $\pi_{4}=k+2,\pi_{5}=k+3$ and so forth. If $\pi_{i}\neq\pi_{i-1}+1$, then $k,1,k+1,\pi_{i},\pi_{i}-1$ would be a copy of 21354. Thus $\pi=k\ 1\ k+1\cdots m\ 2\cdots$. Similar to the above, the sequence $3,\cdots,k-1$ and the sequence $m+1,\cdots,n$ are two increasing sequence of length $k-3$ and $n-m$ respectively. Thus the enumeration can be turned into the number of permutations of the multiset $\{(k-3)\cdot A,(n-m)\cdot B\}$, which is $\binom{n-m+k-3}{k-3}$.

Thus,

[TABLE]

where the third equality follows from the hockey stick identity.

For the enumeration of $|F_{n}^{<k12\cdots\bar{i}>}(321,21354)|$ with $i\geq 3$, we can consider $1\ 2\cdots i-1$ as one letter to play the role of 1, consider $i$ as 2 and consider $k$ as $k-i+2$. Thus

[TABLE]

Therefore,

[TABLE] 4. (4)

$k=n$. Avoiding 321 indicates $2,\cdots,n-1$ to the right of $n$ must be increasing. So

[TABLE]

Thus

[TABLE]

where the forth equality follows from the hockey stick identity.

We have thus proved Theorem 4.3 by induction. ∎