Books versus triangles at the extremal density

TL;DR

This paper explores the relationship between the number of triangles and the maximum number of triangles sharing an edge (book number) in dense graphs, providing new bounds and confirming a conjecture for specific parameter ranges.

Contribution

It proves Mubayi's conjecture about the dependency between triangle count and book number for certain ranges of the parameter 2, advancing understanding of extremal graph configurations.

Findings

Confirmed Mubayi's conjecture for 2=1/6 and 24=0.2495 to 1/4.

Established lower bounds on the number of triangles based on book number constraints.

Connected extremal graph properties with triangle density in dense graphs.

Abstract

A celebrated result of Mantel shows that every graph on $n$ vertices with $\lfloor n^2/4 \rfloor + 1$ edges must contain a triangle. A robust version of this result, due to Rademacher, says that there must in fact be at least $\lfloor n/2 \rfloor$ triangles in any such graph. Another strengthening, due to the combined efforts of many authors starting with Erd\H{o}s, says that any such graph must have an edge which is contained in at least $n/6$ triangles. Following Mubayi, we study the interplay between these two results, that is, between the number of triangles in such graphs and their book number, the largest number of triangles sharing an edge. Among other results, Mubayi showed that for any $1/6 \leq \beta < 1/4$ there is $\gamma > 0$ such that any graph on $n$ vertices with at least $\lfloor n^2/4\rfloor + 1$ edges and book number at most $\beta n$ contains at least $(\gamma -o(1))n^3$ triangles. He also asked for a more precise estimate for $\gamma$ in terms of $\beta$. We make a conjecture about this dependency and prove this conjecture for $\beta = 1/6$ and for $0.2495 \leq \beta < 1/4$, thereby answering Mubayi's question in these ranges.

Equations

$$\frac{1}{n}\sum _{x\in V(G)}e(\overline{N(x)})$$

$$=m+\frac{3t}{n}-\frac{1}{n}\sum _{a\in V(G)}d(a{)}^2\le m+\frac{3t}{n}-\frac{4m^2}{n^2},$$

$$d_A(v)=d(v)-d_C(v)-d_B(v)\ge |A|-|C|-(b+|C|)\ge \frac{n-|C|}{2}-|C|-(b+|C|)=\frac{n}{2}-b-\frac{5}{2}|C|.$$

$$|A||B|+\sum _{v\in C}(d_A(v)+d_B(v))+\left(\genfrac{}{}{0ex}{}{|C|}{2}\right)$$

$$|A||B|-(D-b)d_B(v)+2D|C|+\left(\genfrac{}{}{0ex}{}{|C|}{2}\right)$$

$$s\ge d_B(v)(D(G)-b)\ge \left(\frac{n}{2}-b-\frac{5}{2}|C|\right)(D(G)-b)\ge \left(\frac{n}{4}-\frac{5}{2}|C|\right)(D(G)-b)\ge 2|C|(D(G)-b),$$

$$e_{G_1}(A\cup B,C)\ge \frac{n^2}{4}-|A||B|-\left(\genfrac{}{}{0ex}{}{|C|}{2}\right)\ge \frac{n^2}{4}-{\left(\frac{n-|C|}{2}\right)}^2-\frac{|C{|}^2}{2}=\frac{|C|n}{2}-\frac{3}{4}|C{|}^2.$$

$$d(G_1)\ge \min \left(\frac{n}{4}-\frac{5}{2}|C|,\frac{n}{4}-\frac{3}{8}|C|-1\right)\ge \frac{n}{4}-\frac{5}{2}|C|.$$

$$t^{\prime }(G_1)-t^{\prime }(G)\le s(|C|-d(G_1)).$$

$$\frac{2t^{\prime }(G_1)}{e_{G_1}(A\cup B,C)}\le \frac{2t^{\prime }(G)}{e(A\cup B,C)}$$

$$\left(t^{\prime }(G)-t^{\prime }(G_1)\right)e(A\cup B,C)\ge s{t}^{\prime }(G).$$

$$(d(G_1)-|C|)e(A\cup B,C)\ge t^{\prime }(G).$$

$$|C|{\left(\frac{n}{4}-\frac{5}{2}|C|\right)}^2\ge {\left(\frac{n}{4}\right)}^2|C|\left(1-\frac{20|C|}{n}\right).$$

$$d_B(u,v,G^{\prime })+d_C(u,v,G^{\prime })=d_B(v,G^{\prime })+d_C(u,v,G^{\prime })\ge d_B(v,G^{\prime })+d_C(u,G^{\prime })+d_C(v,G^{\prime })-|C|.$$

$$2\tilde{t}(G^{\prime }):=-|C|e_{G^{\prime }}(A\cup B,C)+\sum _{v\in C}\left(2d_A(v,G^{\prime })d_B(v,G^{\prime })+d_C(v,G^{\prime })d_{A\cup B}(v,G^{\prime })\right)+\sum _{u\in A\cup B}{d}_C(u,G^{\prime }{)}^2.$$

$$|A||B|+|C|2b\le {\left(\frac{n-|C|}{2}\right)}^2+|C|2b=\frac{n^2}{4}+\frac{|C{|}^2}{4}-\frac{ϵ}{2}|C|n\le \frac{n^2}{4},$$

$$-|C|+2d_B(v,G^{\prime \prime })+d_C(v,G^{\prime \prime })+2d_C(u,G^{\prime \prime })+1\le 2|C|+1+2b,$$

$$2\tilde{t}(G_2)$$

$$b^2(n-4b)\le \tilde{t}(G_2)\le \tilde{t}(G_1)\le t(G_1)\le t(G)$$

$$e(C)\ge \frac{n^2}{4}-|A||B|-2b|C|\ge \frac{n^2}{4}-{\left(\frac{n-|C|}{2}\right)}^2-2b|C|=-\frac{|C{|}^2}{4}+\frac{ϵ}{2}|C|n=\frac{|C{|}^2}{4},$$

$$b(G)$$

$$\frac{1}{2b|C|}\left(-3|C{|}^{2}b-6|C|b^2+2|C|bn+4|C{|}^2{b}^2/(n-|C|)\right)=-\frac{3}{2}|C|-3b+n+2|C|b/(n-|C|).$$

$$(6b-n)t(G)\ge b\left(\sum _{v}d(v{)}^2-nm\right).$$

$$(6b-n)t(G)\ge b\left(\sum _v{\left(d(v)-\frac{n}{2}\right)}^2+nm-\frac{n^3}{4}\right)\ge b\sum _v{\left(d(v)-\frac{n}{2}\right)}^2.$$

$$6b-n\ge \frac{3}{m}\sum _v{\left(d(v)-\frac{n}{2}\right)}^2\ge \frac{3}{m}|\{v:|d(v)-\frac{n}{2}|\ge ϵn\}|{ϵ}^2{n}^2.$$

$$d_{M_v}(x,y)$$

$$e(N_v)\ge \frac{1}{2}\left(\frac{1}{2}-ϵ\right)\left(\frac{1}{6}-4ϵ\right)n^2=\left(\frac{1}{24}-O(ϵ)\right)n^2.$$

Full text

1 Introduction

Mantel’s theorem [9] from 1907 is among the earliest results in extremal graph theory. It states that the maximum number of edges that a triangle-free graph on $n$ vertices can have is $\lfloor n^{2}/4\rfloor$, with equality if and only if the graph is the balanced complete bipartite graph. So a graph on $n$ vertices with one more edge must have at least one triangle. Must it have many triangles? Must there be an edge in many triangles? Such questions have a long history of study in extremal graph theory.

In unpublished work, Rademacher answered the first question above in 1950, proving that every graph on $n$ vertices with $\lfloor n^{2}/4\rfloor+1$ edges has at least $\lfloor n/2\rfloor$ triangles, which is tight by adding an edge inside the larger part of a balanced complete bipartite graph. Erdős [3] then extended this result to graphs with a linear number of extra edges and, in [4], studied the problem for larger cliques. Over the last fifty years, many further results in this direction have been obtained by various researchers, see, e.g., [1, 6, 7, 11, 12, 13] and their references.

The second question, about finding an edge in many triangles, was first studied by Erdős [3] in 1962. A book in a graph is a collection of triangles that have an edge in common. The size of the book is the number of such triangles. The book number of a graph $G$, denoted by $b(G)$, is the size of the largest book in the graph. Erdős proved that every graph $G$ on $n$ vertices with $\lfloor n^{2}/4\rfloor+1$ edges satisfies $b(G)\geq n/6-O(1)$ and conjectured that the $O(1)$-term can be removed. Solving this conjecture and answering the second question above, Edwards and, independently, Khadžiivanov and Nikiforov [8] proved that every such graph satisfies $b(G)\geq n/6$, which is tight.

Our concern here is with a problem of Mubayi [10] about the interplay between the two questions above. More precisely, if a graph $G$ on $n$ vertices with $\lfloor n^{2}/4\rfloor+1$ edges satisfies $b(G)\leq b$, at least how many triangles must it have? We write $t(n,b)$ for this minimum number. Mubayi proved that for fixed $\beta\in(1/4,1/2)$, if $b(G)<\beta n$, then $t(n,b)\geq\left(\frac{1}{2}\beta(1-2\beta)-o(1)\right)n^{2}$, a bound which is asymptotically tight. He also showed that $t(n,b)$ changes from quadratic to cubic in $n$ when $b\approx n/4$. More precisely, he proved that for each $\beta\in(1/6,1/4)$ there is $\gamma>0$ such that $t(n,\beta n)\geq\gamma n^{3}$. He then asked for a more precise determination of the optimal $\gamma$ in terms of $\beta$, but added that the problem ‘seems very hard’. Our contribution in this paper is to make a conjecture about this dependency and to confirm this conjecture for $\beta=1/6$ and for $0.2495\leq\beta<1/4$.

To say more, consider the $3$-prism graph, the skeleton of the $3$-prism, consisting of two disjoint triangles with a perfect matching between them. For nonnegative integers $b$ and $n$ with $b\leq n/4$, let $S_{b,n}$ be the graph on $n$ vertices formed by blowing up the $3$-prism graph, where four of the six parts, corresponding to the vertices of two edges of the matching, are of size $b$, and the remaining two parts are of size $\lfloor(n-4b)/2\rfloor$ and $\lceil(n-4b)/2\rceil$. Restated, $S_{b,n}$ has vertex set consisting of six parts $U_{1},U_{2},U_{3},V_{1},V_{2},V_{3}$ with $|U_{1}|=|U_{2}|=|V_{1}|=|V_{2}|=b$, $|U_{3}|=\lfloor(n-4b)/2\rfloor$, $|V_{3}|=\lceil(n-4b)/2\rceil$, $U_{i}$ is complete to $U_{j}$ for $i\not=j$, $V_{i}$ is complete to $V_{j}$ for $i\not=j$, $U_{i}$ is complete to $V_{i}$ for each $i$ and there are no other edges. The graph $S_{b,n}$ has $n$ vertices, $\lfloor n^{2}/4\rfloor$ edges, book number $b$ if $b\geq n/6$ and $b^{2}(n-4b)$ triangles. If $b=0$ or $n/4$, then $S_{b,n}$ is the balanced complete bipartite graph, but otherwise has triangles. We make the following conjecture.111Though the final version of Mubayi’s paper [10] contains no conjecture about the behaviour of $t(n,\beta n)$ for $1/6\leq\beta<1/4$, the original arXiv version described a construction which is almost identical to that given here. As such, we might well ascribe an approximate version of Conjecture 1.1 to him.

Conjecture 1.1**.**

If $n/6\leq b<n/4$, then every graph on $n$ vertices with at least $\lfloor n^{2}/4\rfloor$ edges and book number at most $b$ which is not the balanced complete bipartite graph has at least $b^{2}(n-4b)$ triangles, with equality if and only if the graph is $S_{b,n}$.

Our main result is a proof of Conjecture 1.1 when $b$ is not much smaller than $n/4$.

Theorem 1.2**.**

Conjecture 1.1 holds for graphs with at least $n^{2}/4$ edges if $0.2495n\leq b<n/4$.

While Theorem 1.2 is stated for graphs with at least $n^{2}/4$ edges, the proof is robust enough to yield the analogous result for graphs with $\lfloor n^{2}/4\rfloor$ edges. We only prove the weaker statement for simplicity of presentation.

The perceptive reader will have noticed that our Conjecture 1.1 differs from Mubayi’s question in one small, but important, point of detail: we allow our graphs to have $\lfloor n^{2}/4\rfloor$ edges, whereas Mubayi looks at graphs with at least $\lfloor n^{2}/4\rfloor+1$ edges, thus guaranteeing that there are always some triangles. However, Conjecture 1.1 also implies an asymptotically tight bound on the function $t(n,b)$. To see this, consider a slightly different blow-up of the $3$-prism graph, adding one vertex to each $U_{i}$ and subtracting one vertex from each $V_{i}$. If $n$ is even, we get a graph with book number $b+1$ and with three more edges and $n$ more triangles than $S_{n,b}$. If $n$ is odd, we get a graph with book number $b+1$ and with two more edges and $n-2b$ more triangles than $S_{n,b}$. We now delete two edges, each in $b+1$ triangles but not in a common triangle, if $n$ is even and one edge in $b+1$ triangles if $n$ is odd, yielding the bounds $t(n,b+1)\leq b^{2}(n-4b)+n-2(b+1)$ if $n$ is even and $t(n,b+1)\leq b^{2}(n-4b)+n-2b-(b+1)$ if $n$ is odd. Together with Conjecture 1.1, these constructions imply the required asymptotic estimate on $t(n,b)$ for $n/6\leq b\leq n/4-\omega(1)$, where the $\omega(1)$ term indicates any function tending to infinity with $n$.

We also study what happens at the other end of the range, showing that Conjecture 1.1 holds for $b=n/6$. More precisely, we will make use of results from a paper of Bollobás and Nikiforov [2], themselves derived from the earlier work of Edwards and Khadžiivanov–Nikiforov [8], to show that the conjecture holds in this case.

Theorem 1.3**.**

Conjecture 1.1 holds for graphs with at least $n^{2}/4$ edges if $b=n/6$.

Once again, the theorem holds for graphs with $\lfloor n^{2}/4\rfloor$ edges, but it is more convenient, principally from a notational standpoint, to assume that there are at least $n^{2}/4$ edges.

Notation. For a graph $G$ and vertex $v$, the neighborhood $N(v)$ denotes the set of vertices adjacent to $v$, while the degree of $v$ is denoted by $d(v):=|N(v)|$ and the degree of $v$ into a vertex subset $A$ is denoted by $d_{A}(v):=|N(v)\cap A|$. For two vertices $u$ and $v$, their common neighborhood is denoted by $N(u,v)$ and their codegree $d(u,v)=|N(u,v)|$ is the number of vertices adjacent to both $u$ and $v$. The codegree of $u$ and $v$ into a vertex subset $A$ is denoted by $d_{A}(u,v):=|N(u,v)\cap A|$. For a vertex subset $A$, we write $E(A)$ for the set of edges in $A$ and $e(A)$ for the number of such edges. Similarly, for vertex subsets $A$ and $B$, the set of edges with one vertex in $A$ and the other in $B$ is denoted by $E(A,B)$ and the number of such edges is $e(A,B)=|E(A,B)|$. If the underlying graph $G$ is not clear from context, we include it in the notation.

2 Proof of Theorem 1.2

The following lemma gives a bound on the maximum cut of a graph with few triangles. The result and proof in the special case of triangle-free graphs is due to Erdős, Faudree, Pach and Spencer [5].

Lemma 2.1**.**

If $G$ is a graph with $n$ vertices, $m$ edges and $t$ triangles, then $G$ can be made bipartite by deleting at most $m-\frac{4m^{2}}{n^{2}}+\frac{6t}{n}$ edges.

Proof.

We will show that there is a vertex $x$ for which $N(x)$ and $\overline{N(x)}$ form the desired bipartition of the vertex set by picking $x$ uniformly at random. The expected number of edges in the neighborhood of $x$ is $3t/n$. The expected number of edges in $\overline{N(x)}$ is

[TABLE]

where the first equality follows by double counting the number of triples $(x,a,b)$ of vertices where $(a,b)$ is an edge but $(x,a)$ and $(x,b)$ are not edges and the last inequality is by Cauchy–Schwarz. Thus, the expected number of edges in $N(x)$ and $\overline{N(x)}$ is at most $m+\frac{6t}{n}-\frac{4m^{2}}{n^{2}}$. Hence, there exists a choice of $x$ for which this random variable is at most the expected value. ∎

We use Lemma 2.1 to prove the following result, which gives conditions under which a graph contains a large induced bipartite subgraph.

Lemma 2.2**.**

Let $G$ be a graph with $n$ vertices, $m\geq n^{2}/4$ edges, $t\leq c^{2}n^{3}/24$ triangles and book number $b\leq\left(\frac{1}{2}-c\right)n$. Then $G$ contains an induced bipartite subgraph that contains all but at most $48t/cn^{2}$ vertices.

Proof.

By Lemma 2.1 and $m\geq n^{2}/4$, $G$ has a vertex partition $V(G)=A_{0}\cup B_{0}$ such that all but at most $6t/n\leq c^{2}n^{2}/4$ edges are in $A_{0}\times B_{0}$. We have $|A_{0}|,|B_{0}|\geq\left(1-c\right)n/2$, as otherwise the number of edges in $G$ is at most $|A_{0}||B_{0}|+c^{2}n^{2}/4<n^{2}/4$, a contradiction.

Let $A$ consist of all vertices $a\in A_{0}$ with more than $(b+|B_{0}|)/2$ neighbors in $B_{0}$. The set $A$ is independent, as otherwise we would have an edge in more than $b$ triangles. From each vertex $a\in A_{0}\setminus A$, the number of missing edges to $B_{0}$ is at least $(|B_{0}|-b)/2\geq cn/4$. Thus, we get at least $|A_{0}\setminus A|\cdot cn/4$ missing edges from $A_{0}\setminus A$ to $B_{0}$. We also have that the number of missing edges across $A_{0}\times B_{0}$ is at most $|A_{0}||B_{0}|-(m-6t/n)\leq 6t/n$, so it follows that $|A_{0}\setminus A|\leq(6t/n)/(cn/4)=24t/cn^{2}$. Similarly, letting $B$ consist of all vertices $b\in B_{0}$ with more than $(b+|A_{0}|)/2$ neighbors in $B_{0}$, we have that $B$ is independent and $|B_{0}\setminus B|\leq 24t/cn^{2}$. Thus, $A\cup B$ induces a bipartite subgraph that contains all but at most $48t/cn^{2}$ vertices. ∎

Our goal in this section is to prove Theorem 1.2. Since the proof is somewhat long, we first give an outline. Let $G$ be a graph on $n$ vertices with at least $n^{2}/4$ edges and book number at most $b$ (where $b<n/4$, but is not much smaller) which is not the balanced complete bipartite graph, but contains as few triangles as possible. We let $H$ be an induced bipartite subgraph of $G$ with the maximum number of vertices. Let $A$ and $B$ be the parts of $H$ and let $C$ be the remaining vertices, so that $A$, $B$ and $C$ form a vertex partition of $G$. We begin the proof by deriving some simple properties of the graph $G$. For instance, as there are not many triangles in $G$, we can use Lemma 2.2 to deduce that $|C|$ is small. We can also conclude that $C$ is nonempty since $G$ has at least $n^{2}/4$ edges but is not complete bipartite. Moreover, by the choice of $H$, every vertex in $C$ has a neighbor in both $A$ and $B$. With a little more effort, we can even show that the degree of each vertex in $C$ is at least the maximum of $A$ and $B$.

From this point on, we do not need to use the fact that each edge of $G$ is in at most $b$ triangles, just that a random edge from $E(A\cup B,C)$ is in expectation in at most $b$ triangles. We form a new graph $G_{1}$ on the same vertex set as $G$ by adding edges to $A\times B$ to make $A$ complete to $B$ and deleting the same number of edges from $E(A\cup B,C)$. We can do this so that in $G_{1}$ each vertex in $C$ has degree at most $b$ to $A$ and degree at most $b$ to $B$, the total number of triangles does not increase and a random edge in $(A\cup B)\times C$ is in expectation in at most $b$ triangles. We are not able to guarantee that $G_{1}$ has book number at most $b$, but tracking this related expectation is sufficient for our purposes.

We now form another graph $G_{2}$ from $G_{1}$ by deleting edges in $E(C)$ and adding an equal number of edges to $(A\cup B)\times C$ so that each vertex in $C$ has degree $b$ to $A$ and degree $b$ to $B$. There are three types of triangle in $G_{2}$, those with exactly $i$ vertices in $C$ for $i=1,2,3$. It is easy to compute a lower bound on the number of type $1$ or $2$ triangles in $G_{2}$ and we show that this also gives a lower bound in $G_{1}$. Furthermore, the expected number of triangles containing a random edge of $E(G_{2})\cap(A\cup B)\times C$ is at most the expected number of triangles containing a random edge of $E(G_{1})\cap(A\cup B)\times C$. If $|C|<n-4b$, this expected number is larger than $b$, contradicting the fact that the corresponding expected number in $G$ is at most $b$. If $|C|\geq n-4b$, we find that the number of type $1$ or $2$ triangles in $G_{2}$ (and, hence, in $G$) is at least $b^{2}(n-4b)$, with equality only if $|C|=n-4b$. Furthermore, equality can occur only if $G=G_{2}$ and all triangles are of type $1$, so no edge in $C$ is in a triangle. But equality also implies that $|E(C)|\geq|C|^{2}/4$, so Mantel’s theorem forces $C$ to induce a balanced complete bipartite graph. The parts of this partition determine two parts of the graph $S_{b,n}$, while the set of neighbors and nonneighbors of any vertex in $C$ partition each of $A$ and $B$ into two pieces, determining the remaining parts.

Proof of Theorem 1.2. Let $G$ be a graph on $n$ vertices with $m\geq n^{2}/4$ edges and book number at most $b=(1-\epsilon)n/4$, where $\epsilon\leq 1/500$, which is not the balanced complete bipartite graph, but for which the number $t$ of triangles is as small as possible. As the graph $S_{b,n}$ satisfies all of these conditions except possibly the last and has $b^{2}(n-4b)$ triangles, we may assume that $t\leq b^{2}(n-4b)\leq\epsilon n^{3}/16$.

Let $H$ be the largest induced bipartite subgraph of $G$ and let $A$ and $B$ denote the parts of $H$ with $|A|\geq|B|$. Let $C=V(G)\setminus V(H)$. If a vertex in $C$ is not adjacent to some vertex in $A$, then we can add it to $A$ and get a larger induced bipartite subgraph of $G$, a contradiction. Since similar reasoning holds with $B$ in place of $A$, we have the following claim.

Claim 1: Every vertex in $C$ has a neighbor in both $A$ and $B$.

By Lemma 2.2 with $c=1/4$, we have the next claim.

Claim 2: $|C|\leq 192t/n^{2}\leq 12\epsilon n$.

If $|C|=0$, then $G$ is bipartite and, as the number of edges is at least $n^{2}/4$, $G$ has to be the balanced complete bipartite graph, a contradiction which yields the following claim.

Claim 3: The set $C$ is nonempty.

We next observe that $G$ must have large minimum degree.

Claim 4: Every vertex in $B\cup C$ has degree at least $|A|$ and every vertex in $A$ has degree at least $|B|$.

Proof: Suppose that $v\in B\cup C$. If $d(v)<|A|$, we can delete all edges containing $v$ and then make $v$ complete to $A$. This operation increases the number of edges of $G$ and, as $v$ is not in any triangle in the new graph, does not increase $b(G)$ or $t(G)$. We can then delete an edge of the resulting graph which is in a triangle, obtaining a new graph $G^{\prime}$ with at least $n^{2}/4$ edges which still has $b(G^{\prime})\leq b$ but has fewer triangles than $G$. If $G^{\prime}$ has zero triangles, then it is the complete balanced bipartite graph on an even number of vertices and the deleted edge would be in $n/2$ triangles, contradicting that the book number is at most $n/4$. Otherwise, $G^{\prime}$ contradicts the choice of $G$ and the claim follows. The case where $v\in A$ follows similarly. ∎

As $A$ is an independent set with minimum degree at least $|B|$, each vertex $u\in A$ is adjacent to all but at most $|C|$ vertices in $B$. Similarly, every vertex in $B$ is adjacent to all but at most $|C|$ vertices in $A$. We thus have the following claim.

Claim 5: Every vertex in $A$ (respectively, $B$) is adjacent to all but at most $|C|$ vertices in $B$ (respectively, $A$).

From Claims 1 and 5, we have the following claim, as otherwise $v$ is in an edge in more than $b$ triangles.

Claim 6: For every vertex $v\in C$, $d_{A}(v),d_{B}(v)\leq b+|C|$.

From the previous claim, for each vertex $v\in C$, we have

[TABLE]

Since the same bound clearly holds for $d_{B}(v)$, we have the following claim.

Claim 7: For every vertex $v\in C$, $d_{A}(v),d_{B}(v)\geq\frac{n}{2}-b-\frac{5}{2}|C|$.

Let $D=D(G)=\max_{v\in C}(d_{A}(v),d_{B}(v))$ and $d=d(G)=\min_{v\in C}(d_{A}(v),d_{B}(v))$ so that $d\leq d_{A}(v),d_{B}(v)\leq D$ for all vertices $v\in C$. In general, for a graph parameter, we will usually not specify the graph if it is $G$, but we will if it is another graph, as we did in the proof of Claim 4.

Claim 8: $|C|>\frac{2}{3}\left(n-4b\right)$.

Proof: Suppose otherwise, that $|C|\leq\frac{2}{3}\left(n-4b\right)$. If $D\leq b$, then the total number of edges of $G$ is at most

[TABLE]

a contradiction. Thus, we must have $D>b$. Suppose $D=d_{A}(v)$ with $v\in C$ (the case $D=d_{B}(v)$ is handled in the same way). For each $u\in N_{B}(v)$, as the edge $(u,v)$ is in at most $b$ triangles, there must be at least $D-b$ missing edges from $u$ to $N_{A}(v)$. Hence, there are at least $(D-b)d_{B}(v)$ missing edges between $A$ and $B$. Then the number of edges in $G$ is at most

[TABLE]

a contradiction. The first inequality above uses Claim 7, while the second inequality uses $D-b>0$ and $\frac{n}{2}-b-\frac{9}{2}|C|>0$, which follows from $b\leq\frac{n}{4}$, Claim 2 and $\epsilon<1/216$. ∎

For $i\in\{0,1,2,3\}$, we say that a triangle in $G$ is of type $i$ if it contains exactly $i$ vertices from $C$. We let $t_{i}$ denote the number of triangles of type $i$. As there are no triangles in $H=G[A\cup B]$, we have $t_{0}=0$. Let $t^{\prime}=t_{1}+t_{2}$ be the number of triangles of type 1 or 2.

Let $\bar{b}(G)$ denote the expected number of triangles containing a random edge in $E(A\cup B,C)$. That is, $\bar{b}(G)=2t^{\prime}(G)/e_{G}(A\cup B,C)$.

Claim 9: There is a graph $G_{1}$ with $V(G_{1})=V(G)$ and $e(G_{1})=e(G)$ such that $G_{1}$ induces a complete bipartite graph on $A\cup B$ with parts $A$ and $B$, $D(G_{1})\leq b$, $d(G_{1})\geq\frac{n}{4}-\frac{5}{2}|C|$, $t(G_{1})\leq t(G)$, $t^{\prime}(G_{1})\leq t^{\prime}(G)$ and $\bar{b}(G_{1})\leq\bar{b}(G)$. Moreover, if $G_{1}\not=G$, then $t(G_{1})<t(G)$.

Proof: Suppose there are $s$ missing edges between $A$ and $B$ in $G$. Consider adding all $s$ missing edges between $A$ and $B$ (so $A$ is now complete to $B$) and then deleting $s$ edges between $C$ and $A\cup B$, deleting them one at a time from a vertex in $C$ of largest degree to $A$ or $B$ to obtain a new graph $G_{1}$. To see that this process is possible, note that each vertex $v\in B$ has degree at least $|A|$ by Claim 4 and so has at least as many neighbors in $C$ as it has nonneighbors in $A$. Note, by construction, that $V(G_{1})=V(G)$ and $e(G_{1})=e(G)$.

If $D(G)>b$ and $v$ is a vertex with $d_{A}(v)=D(G)$ (the case $d_{B}(v)=D(G)$ is handled in the same way), then, in the graph $G$, for each $u\in N_{B}(v)$, the edge $(u,v)$ is in at most $b$ triangles, so $u$ has at least $D(G)-b$ missing edges to $A$. Thus, by Claim 7,

[TABLE]

where the last inequality follows from Claim 2 and $\epsilon<1/216$. The final expression is an upper bound on the number of edges that must be deleted between $C$ and $A\cup B$ to guarantee $\max_{v\in C}(d_{A}(v),d_{B}(v))\leq b$. We thus have $D(G_{1})\leq b$ in this case. If $D(G)\leq b$, then, since we only deleted edges between $C$ and $A\cup B$ to make $G_{1}$, $D(G_{1})\leq D(G)\leq b$. Hence, in either case, we have $D(G_{1})\leq b$.

Observe that if $D(G_{1})>d(G_{1})+1$, then we must have $d(G_{1})=d(G)$ as if, say, $d_{A}(v)=d(G)$, we would never delete an edge from $v$ to $A$ in the process of obtaining $G_{1}$. In this case, we have, by Claim 7, that $d(G_{1})=d(G)\geq\frac{n}{2}-b-\frac{5}{2}|C|\geq\frac{n}{4}-\frac{5}{2}|C|$. Otherwise, we have that the degrees $d_{A}(v)$, $d_{B}(v)$ in $G_{1}$ are all simply the average degree rounded up or down. The number of edges of $G_{1}$ between $C$ and $A\cup B$ satisfies

[TABLE]

So the average value of $d_{X}(v)$ over all $2|C|$ choices of $v\in C$ and $X\in\{A,B\}$ is at least $\frac{n}{4}-\frac{3}{8}|C|$. Hence,

[TABLE]

Since each of the $s$ edges added between $A$ and $B$ is in at most $|C|$ triangles, in total this process added at most $s|C|$ triangles. Once these edges have been added and the graph between $A$ and $B$ is complete bipartite, we remove the $s$ edges from between $C$ and $A\cup B$. Since each such edge is contained in at least $d(G_{1})$ triangles, we remove at least $sd(G_{1})$ triangles in total. Hence,

[TABLE]

As $n\geq 14\cdot 12\epsilon n>14|C|$ for $\epsilon<1/168$, it follows from (1) that $t^{\prime}(G_{1})\leq t^{\prime}(G)$. As no edges in $C$ are added or deleted in obtaining $G_{1}$ from $G$, we have $t_{3}(G_{1})=t_{3}(G)$ and, hence, $t(G_{1})\leq t(G)$. Moreover, if $s\not=0$, then $t^{\prime}(G_{1})<t^{\prime}(G)$ and, hence, $t(G_{1})<t(G)$.

Finally, we check that $\bar{b}(G_{1})\leq\bar{b}(G)$. This is equivalent to showing that

[TABLE]

and, as $e_{G_{1}}(A\cup B,C)=e(A\cup B,C)-s$, this is equivalent to showing that

[TABLE]

From the bound $t^{\prime}(G)-t^{\prime}(G_{1})\geq s(d(G_{1})-|C|)$, this would follow if we could show that

[TABLE]

Each edge in $E(A\cup B,C)$ is in at most $b$ triangles in $G$ and each type $1$ or $2$ triangle has exactly two such edges, so $t^{\prime}(G)\leq e(A\cup B,C)b/2$. Hence, it suffices to show that $d(G_{1})-|C|\geq b/2$, which follows from (1), $|C|\leq 12\epsilon n$, $b\leq n/4$ and $\epsilon\leq 1/336$. This completes the proof of Claim 9. ∎

Claim 10: $|C|\leq\frac{1}{1-240\epsilon}(n-4b)\leq 2(n-4b)=2\epsilon n$.

Proof: Suppose, for the sake of contradiction, that $|C|>\frac{1}{1-240\epsilon}(n-4b)$. Each vertex $v\in C$ is in $d_{A}(v,G_{1})d_{B}(v,G_{1})$ type $1$ triangles in $G_{1}$. As $d_{A}(v,G_{1})d_{B}(v,G_{1})\geq d(G_{1})^{2}\geq\left(\frac{n}{4}-\frac{5}{2}|C|\right)^{2}$ by Claim 9, the number of triangles in $G_{1}$ is at least

[TABLE]

This last expression is an increasing function of $|C|$ for $|C|\leq\frac{n}{40}$ (which holds since $|C|\leq 12\epsilon n$ and $\epsilon\leq 1/480$). Hence, as $b\leq n/4$ and $\frac{1}{1-240\epsilon}(n-4b)<|C|\leq 12\epsilon n$, we have that the number of triangles in $G_{1}$ (and, hence, $G$) is larger than $b^{2}(n-4b)$, a contradiction. ∎

For any graph $G^{\prime}$ on $V(G)$ for which $A\cup B$ induces a complete bipartite graph, the number of triangles containing an edge $(u,v)\in E_{G^{\prime}}(A,C)$ is

[TABLE]

Similarly, if $(u,v)\in E_{G^{\prime}}(B,C)$, then the number of triangles in $G^{\prime}$ containing the edge $(u,v)$ is at least $d_{A}(v,G^{\prime})+d_{C}(u,G^{\prime})+d_{C}(v,G^{\prime})-|C|$. Summing over all edges in $E_{G^{\prime}}(A\cup B,C)$ and using the fact that each type 1 or 2 triangle contains exactly two such edges, the number of type 1 or 2 triangles in $G^{\prime}$ is at least $\tilde{t}(G^{\prime})$, defined by

[TABLE]

To see this, note, for example, that each term of the form $d_{C}(u,G^{\prime})$ appears $d_{C}(u,G^{\prime})$ times, once for each edge $(u,v)\in E_{G^{\prime}}(A\cup B,C)$.

Claim 11: There is a graph $G_{2}$ obtained from $G_{1}$ by deleting some edges with both vertices in $C$ and adding an equal number of edges to $(A\cup B)\times C$ such that $d_{A}(v,G_{2})=d_{B}(v,G_{2})=b$ for all $v\in C$, $\tilde{t}(G_{2})\leq\tilde{t}(G_{1})$ and $e_{G_{2}}(A\cup B,C)\geq e_{G_{1}}(A\cup B,C)$. Moreover, if $G_{2}\not=G_{1}$, then $\tilde{t}(G_{2})<\tilde{t}(G_{1})$.

Proof: As $d_{A}(v,G_{1}),d_{B}(v,G_{1})\leq b$, we can arbitrarily delete edges from $C$ (as long as there are edges) and add an equal number of edges to $(A\cup B)\times C$ to obtain the graph $G_{2}$ with $d_{A}(v,G_{2})=d_{B}(v,G_{2})=b$. This is possible because, by Claim 10 and $b=(1-\epsilon)n/4$, the number of edges we would get, not including those in $C$, is

[TABLE]

leaving enough room for a nonnegative number of edges in $C$. Note that, by construction, $G_{2}$ has at least as many edges across $(A\cup B)\times C$ as $G_{1}$.

Let $G^{\prime}$ be a graph obtained at some stage of the process of transforming $G_{1}$ into $G_{2}$. If we delete an edge $(v,v^{\prime})$ from $G^{\prime}$ with $v,v^{\prime}\in C$ to obtain $G^{\prime\prime}$, then it decreases the value of $2\tilde{t}(G^{\prime})$ by $d_{A\cup B}(v,G^{\prime})+d_{A\cup B}(v^{\prime},G^{\prime})\geq 4\left(\frac{n}{4}-\frac{5}{2}|C|\right)=n-10|C|$, where the inequality is by the lower bound on $d(G_{1})$ from Claim 9. If we add an edge $(u,v)\in(A\cup B)\times C$ to this graph (with, say, $u\in A$), it increases the value of $2\tilde{t}(G^{\prime\prime})$ by

[TABLE]

where the last inequality uses $d_{C}(v,G^{\prime\prime}),d_{C}(u,G^{\prime\prime})\leq|C|$ and $d_{B}(v,G^{\prime\prime})\leq b$. Hence, in deleting an edge with both vertices in $C$ and adding an edge in $(A\cup B)\times C$, we decreased the value of $\tilde{t}(G^{\prime})$ by at least $n-10|C|-(2|C|+1+2b)\geq\frac{n}{2}-13|C|\geq\frac{n}{2}-156\epsilon n>0$, where we used Claim 2. Thus, in the process of going from $G_{1}$ to $G_{2}$, $\tilde{t}$ decreases at each step, so $\tilde{t}(G_{2})\leq\tilde{t}(G_{1})$, with equality only if $G_{2}=G_{1}$. ∎

We have

[TABLE]

where, in the first inequality, we used the Cauchy–Schwarz inequality and $\sum_{u\in A\cup B}d_{C}(u,G_{2})=2|C|b$. The last expression, as a function of $|C|$, is increasing for $b$ in the range of interest and $|C|$ in the range determined by Claims 8 and 10, which can be seen by taking the derivative with respect to $|C|$. Using this fact, we may evaluate this expression at $|C|=n-4b$ to conclude that

[TABLE]

for $|C|\geq n-4b$. Furthermore, the only way we could get equality in the above bound is if $|C|=n-4b$, $|A|=|B|=2b$ and if we moved no edges in getting $G_{1}$ from $G$ and $G_{2}$ from $G_{1}$, so that $G_{2}$ and $G$ are the same. Therefore, in $G$, $A$ is complete to $B$ and $d_{A}(v)=d_{B}(v)=b$ for each vertex $v\in C$. Hence, as each vertex in $C$ is in $b^{2}$ type $1$ triangles, the number of triangles of type $1$ in $G$ is $b^{2}(n-4b)$ so there are no type $2$ or $3$ triangles in $G$. In particular, no edge in $C$ belongs to a triangle. On the other hand,

[TABLE]

where, in the last inequality, we used that $|C|=n-4b=\epsilon n$. As $C$ has at least $|C|^{2}/4$ edges but induces a triangle-free graph, Mantel’s theorem implies that $|C|$ is even (which is equivalent to $n$ being even) and $C$ induces a balanced complete bipartite graph with parts $C_{1}$, $C_{2}$ of equal size. As no edge in $C$ is in a triangle with a vertex in $A$ or $B$ and yet $d_{A}(v)=b=|A|/2$ and $d_{B}(v)=b=|B|/2$ for each $v\in C$, we have equitable partitions $A=A_{1}\cup A_{2}$ and $B=B_{1}\cup B_{2}$ such that $C_{1}$ is complete to $A_{1}\cup B_{1}$, $C_{2}$ is complete to $A_{2}\cup B_{2}$ and there are no other edges between $A\cup B$ and $C$. It is now easy to check that $G$ is the graph $S_{n,b}$ with parts $A_{1},B_{1},C_{1},B_{2},A_{2},C_{2}$.

It remains to check the case $|C|<n-4b$. We will show that there is an edge in more than $b$ triangles, a contradiction. Indeed,

[TABLE]

This last expression is at least

[TABLE]

In the range of interest, this function is strictly decreasing in $|C|$. Given that we are assuming that $|C|<n-4b$, if we evaluate the above expression at $|C|=n-4b$, we get $b$ and, hence, $b(G)$ is greater than this value, a contradiction. This completes the proof of Theorem 1.2. ∎

3 Proof of Theorem 1.3

The main result of this section, which easily implies Theorem 1.3, is as follows.

Theorem 3.1**.**

If $\epsilon>0$ is sufficiently small, then every graph on $n$ vertices with at least $n^{2}/4$ edges and book number at most $\left(\frac{1}{6}+\epsilon^{3}\right)n$ which is not the balanced complete bipartite graph has at least $\left(\frac{1}{108}-O(\epsilon)\right)n^{3}$ triangles.

Proof.

Suppose that $G$ is a graph satisfying the assumptions of the theorem. As with Theorem 1.2, we will prove the result through a sequence of claims.

Claim A: $G$ is approximately regular, in that $|\{v:\left|d(v)-\frac{n}{2}\right|\geq\epsilon n\}|\leq\epsilon n$.

Proof: Let $b=b(G)$, $t(G)$ be the number of triangles in $G$ and $m$ be the number of edges. We use the following inequality, proved by Bollobás and Nikiforov [2] (see Equation (8)),

[TABLE]

Since $\sum_{v}d(v)^{2}=\sum_{v}\left(d(v)-\frac{n}{2}\right)^{2}+2mn-\frac{n^{3}}{4}$, we have

[TABLE]

As the right-hand side is non-negative and $t(G)>0$ (since $G$ is not the balanced complete bipartite graph), it follows that $6b-n\geq 0$. Using the simple bound $t(G)\leq\frac{1}{3}bm$, we find that

[TABLE]

Suppose now that $|\{v:\left|d(v)-\frac{n}{2}\right|\geq\epsilon n\}|>\epsilon n$. Substituting this in and using $m\leq n^{2}/2$ yields $6b-n>\frac{3\epsilon^{3}n^{3}}{m}\geq 6\epsilon^{3}n$ and, hence, $b>\left(\frac{1}{6}+\epsilon^{3}\right)n$, a contradiction. ∎

Now remove any vertices of degree less than $\left(\frac{1}{2}-\epsilon\right)n$ from $G$. By Claim A, this gives a new graph $G^{\prime}$ on $n^{\prime}\geq(1-\epsilon)n$ vertices. Since $G$ had at least $n^{2}/4$ edges and we removed at most $(n-n^{\prime})(\frac{1}{2}-\epsilon)n$ edges, $G^{\prime}$ also has at least $(n^{\prime})^{2}/4$ edges. The minimum degree of $G^{\prime}$ is at least $(\frac{1}{2}-2\epsilon)n\geq(\frac{1}{2}-2\epsilon)n^{\prime}$ and $b(G^{\prime})\leq(\frac{1}{6}+\epsilon^{3})n\leq(\frac{1}{6}+\frac{\epsilon}{5})n^{\prime}$. For simplicity, we shall again call this smaller graph $G$ and suppose that it has $n$ vertices. Furthermore, increasing $\epsilon$ by at most a factor $2$, we have that the minimum degree of $G$ is at least $(\frac{1}{2}-\epsilon)n$ and $b(G)\leq(\frac{1}{6}+\frac{\epsilon}{10})n$. The additional error introduced by increasing $\epsilon$ is easily covered by the $O(\epsilon)$ term in our bound on the number of triangles.

Given any vertex $v\in G$, we will use the shorthand $N_{v}$ for the neighbors of $v$ and $M_{v}$ for the set of nonneighbors (including $v$). Note that we have $|N_{v}|\geq\left(\frac{1}{2}-\epsilon\right)n$ and $|M_{v}|\leq\left(\frac{1}{2}+\epsilon\right)n$ for all $v$. Clearly, for any $v\in G$ and $x\in N_{v}$, we have $d_{N_{v}}(x)\leq b(G)\leq\left(\frac{1}{6}+\frac{\epsilon}{10}\right)n$.

Claim B: Given $v\in G$ and $x\in N_{v}$, if $d_{N_{v}}(x)\neq 0$, then $\left(\frac{1}{6}-4\epsilon\right)n\leq d_{N_{v}}(x)\leq\left(\frac{1}{6}+\frac{\epsilon}{10}\right)n$.

Proof: Let $y\in N_{v}$ be a neighbour of $x$. Note that $x$ and $y$ both have degree at least $\left(\frac{1}{2}-\epsilon\right)n$. Thus, the number of common neighbors in $M_{v}$ is at least

[TABLE]

where we used that $d_{M_{v}}(x)=d(x)-d_{N_{v}}(x)\geq\left(\frac{1}{2}-\epsilon\right)n-d_{N_{v}}(x)$ and $|M_{v}|\leq\left(\frac{1}{2}+\epsilon\right)n$. Using the bounds $d_{M_{v}}(x,y)\leq d(x,y)\leq b(G)\leq\left(\frac{1}{6}+\frac{\epsilon}{10}\right)n$ and $d_{N_{v}}(y)\leq b(G)\leq\left(\frac{1}{6}+\frac{\epsilon}{10}\right)n$, we deduce that $d_{N_{v}}(x)\geq\left(\frac{1}{6}-3\epsilon-\frac{\epsilon}{5}\right)n\geq\left(\frac{1}{6}-4\epsilon\right)n$. ∎

This proof also shows that if $x$ and $y$ are neighbors in $N_{v}$, then they have at least $\left(\frac{1}{6}-4\epsilon\right)n$ common neighbors in $M_{v}$. Therefore, they can have at most $O(\epsilon n)$ common neighbors in $N_{v}$. Note also that we must have $d(v)=|N_{v}|\leq\left(\frac{1}{2}+O(\epsilon)\right)n$, since a smaller bound on the size of $|M_{v}|$ would force $d_{M_{v}}(x,y)>\left(\frac{1}{6}+\frac{\epsilon}{10}\right)n$.

Claim C: Suppose $e(N_{v})>0$. Then $e(N_{v})\geq\left(\frac{1}{36}-O(\epsilon)\right)n^{2}$ and there are at least $\left(\frac{1}{216}-O(\epsilon)\right)n^{3}$ triangles with two vertices in $N_{v}$ and one in $M_{v}$.

Proof: Suppose $x\sim y$ in $N_{v}$. Then $d_{N_{v}}(x)>0$, so $d_{N_{v}}(x)\geq\left(\frac{1}{6}-4\epsilon\right)n$ and similarly for $y$. Moreover, since $x$ and $y$ have at most $O(\epsilon n)$ common neighbors in $N_{v}$, the neighbors of $x$ and the neighbors of $y$ in $N_{v}$ give at least $\left(\frac{1}{3}-O(\epsilon)\right)n$ vertices of positive degree in $G[N_{v}]$, each of which has degree at least $\left(\frac{1}{6}-4\epsilon\right)n$. Thus, $e(N_{v})\geq\frac{1}{2}\left(\frac{1}{3}-O(\epsilon)\right)\left(\frac{1}{6}-4\epsilon\right)n^{2}=\left(\frac{1}{36}-O(\epsilon)\right)n^{2}$. Moreover, each of these edges must have at least $\left(\frac{1}{6}-O(\epsilon)\right)n$ common neighbors in $M_{v}$, giving $\left(\frac{1}{36}-O(\epsilon)\right)\left(\frac{1}{6}-O(\epsilon)\right)n^{3}=\left(\frac{1}{216}-O(\epsilon)\right)n^{3}$ triangles with two vertices in $N_{v}$ and one in $M_{v}$. ∎

Claim D: For every $v\in G$, $e(N_{v})=e(M_{v})+O(\epsilon n^{2})$.

Proof: We have $\sum_{x\in N_{v}}d(x)=2e(N_{v})+e(N_{v},M_{v})$ and $\sum_{x\in M_{v}}d(x)=2e(M_{v})+e(N_{v},M_{v})$. Since the graph is almost regular by Claim A and $|N_{v}|,|M_{v}|=\left(\frac{1}{2}+O(\epsilon)\right)n$, it follows that the two sums are approximately equal, that is, $e(N_{v})=e(M_{v})+O(\epsilon n^{2})$. ∎

Claim E: For every $v\in G$, there is some $w\in N_{v}$ with $d_{N_{v}}(w)=0$.

Proof: Suppose on the contrary that $d_{N_{v}}(w)>0$ for all $w\in N_{v}$. Then, by $|N_{v}|\geq\left(\frac{1}{2}-\epsilon\right)n$ and Claim B,

[TABLE]

Each of these edges extends to at least $\left(\frac{1}{6}-O(\epsilon)\right)n$ triangles with a vertex in $M_{v}$, giving at least $\left(\frac{1}{144}-O(\epsilon)\right)n^{3}$ such triangles. Moreover, by Claim D, we have $e(M_{v})\geq\left(\frac{1}{24}-O(\epsilon)\right)n^{2}$.

Now consider any edge $x\sim y$ in $N_{v}$ and count the number of triangles containing $x$ or $y$ with two vertices in $M_{v}$. By Claim B, we have $|N_{M_{v}}(x)|\leq\left(\frac{1}{3}+O(\epsilon)\right)n$. Together with $d_{M_{v}}(x,y)\geq\left(\frac{1}{6}-4\epsilon\right)n$, this implies that $|N_{M_{v}}(x)\setminus N_{M_{v}}(y)|\leq\left(\frac{1}{6}+O(\epsilon)\right)n$ and similarly for $|N_{M_{v}}(y)\setminus N_{M_{v}}(x)|$. The proof of Claim B also implies that $|M_{v}\setminus(N_{M_{v}}(x)\cup N_{M_{v}}(y))|=O(\epsilon n)$, otherwise, from inequality (2), we have that $d_{M_{v}}(x,y)$ is too big, a contradiction. Therefore, there are at most $\left(\frac{1}{36}+O(\epsilon)\right)n^{2}$ edges in $M_{v}$ that do not form a triangle with $x$ or $y$. This leaves at least $\left(\frac{1}{72}-O(\epsilon)\right)n^{2}$ edges that do form a triangle with at least one of $x$ or $y$. Summing these up for every edge in $N_{v}$ gives $\left(\frac{1}{1728}-O(\epsilon)\right)n^{4}$. By Claim B, each vertex $x$ has degree at most $\left(\frac{1}{6}+\frac{\epsilon}{10}\right)n$ in $N_{v}$, so this gives an upper bound on the number of times any triangle can be counted. Hence, the total number of such triangles (one vertex in $N_{v}$, two in $M_{v}$) is at least $\left(\frac{1}{288}-O(\epsilon)\right)n^{3}$. This implies that the total number of triangles in $G$ is at least $\left(\frac{1}{96}-O(\epsilon)\right)n^{3}$, which is too large. ∎

We may now complete the proof. Since $e(G)\geq n^{2}/4$ and $G$ is not the balanced complete bipartite graph, $G$ must contain a triangle. Let $v$ be a vertex of this triangle, so that $e(N_{v})>0$. By Claim C, we have $e(N_{v})\geq\left(\frac{1}{36}-O(\epsilon)\right)n^{2}$ and at least $\left(\frac{1}{216}-O(\epsilon)\right)n^{3}$ triangles with two vertices in $N_{v}$. Moreover, by Claim E, there is some $w\in N_{v}$ with $d_{N_{v}}(w)=0$, which implies that $N_{w}\subset M_{v}$. Note now that $|M_{v}\setminus N_{w}|=O(\epsilon n)$. Since $e(M_{v})=e(N_{v})+O(\epsilon n^{2})$ by Claim D, it follows that $e(N_{w})\geq e(N_{v})-O(\epsilon n^{2})>0$. Hence, again by Claim C, there are at least $\left(\frac{1}{216}-O(\epsilon)\right)n^{3}$ triangles with two vertices in $N_{w}$. Since $N_{w}\cap N_{v}=\emptyset$, these triangles are distinct from those above, which gives $\left(\frac{1}{108}-O(\epsilon)\right)n^{3}$ triangles in total. ∎

If $\epsilon=0$ and equality holds throughout the argument above, consider a vertex $v$ which is contained in a triangle and a vertex $x\in N_{v}$ with $d_{N_{v}}(x)\neq 0$. Then, by Claim A, the graph is $n/2$-regular and, by Claim B, we have $d_{N_{v}}(x)=n/6$. Note, moreover, that $N_{v}$ is triangle-free by the comments after Claim B, which implies that $N(v,x)$ is an independent set. Similarly, for any $y\in N(v,x)$, $d_{N_{v}}(y)=n/6$ and $N(v,y)$ must be an independent set. We now split $N_{v}$ into three parts, each with $n/6$ vertices, namely, $N(v,x)$, $N(v,y)$ and the remainder, which we label $R_{v}$. By the proof of Claim C, we see that if $e(N_{v})>n^{2}/36$, then there are more than $n^{3}/216$ triangles with two vertices in $N_{v}$ and one in $M_{v}$. Since, by Claim E, there is a vertex $w\in N_{v}$ with no neighbors in $N_{v}$, we have that $N_{w}=M_{v}$ and, hence, there are at least $n^{3}/216$ triangles with two vertices in $M_{v}$ and one in $N_{v}$. So altogether there are more than $n^{3}/108$ triangles, a contradiction. This implies that $e(N_{v})=n^{2}/36$ and, therefore, there are exactly $n/3$ vertices in $N_{v}$ with degree $n/6$ in $N_{v}$. Since the neighbors of $x$ and $y$ must all have positive degree in $N_{v}$ (which by the above discussion should be $n/6$), we conclude that the vertices in $R_{v}$ have no neighbors in $N_{v}$, while there must be a complete bipartite graph between $N(v,x)$ and $N(v,y)$.

Picking now any vertex $u\in R_{v}$, we see that its neighborhood must be $M_{v}$, the complement of $N_{v}$. By the same argument as above, the induced graph on $M_{v}=N_{u}$ must consist of a balanced complete bipartite graph between two parts $N(u,x^{\prime}),N(u,y^{\prime})$, each with $n/6$ vertices, and a set $R_{u}$ of $n/6$ vertices with no neighbors in $M_{v}$, each of which must then be complete to $N_{v}$. Since there are $n^{3}/216$ triangles between $R_{u},N(v,x)$ and $N(v,y)$ and a similar number between $R_{v},N(u,x^{\prime})$ and $N(u,y^{\prime})$, we see that there are no more triangles, so any vertex in $N(v,x)\cup N(v,y)$ can only have neighbors in one of $N(u,x^{\prime})$ or $N(u,y^{\prime})$ and vice versa. Putting all this together, we see that equality holds only if the graph is the blow-up of a $3$-prism with $n/6$ vertices in each part, as claimed.

4 Concluding remarks

The most obvious question that we have left open is Conjecture 1.1. Our results only establish this conjecture when $b=n/6$ or when $0.2495n\leq b<n/4$, so much more remains to be done. In the first instance, it might be interesting to show that there is some $\epsilon>0$ such that the conjecture holds for all $n/6\leq b\leq\left(\frac{1}{6}+\epsilon\right)n$.

There are of course many natural variants of Mubayi’s question: how does the tradeoff between triangles and books change if we assume there are at least $\alpha n^{2}$ edges for some $1/4<\alpha<1/2$? what happens for larger cliques? what about hypergraphs? But the question also points to a more general metaquestion, of how the local and global counts for substructures play off against one another. There are many contexts besides graphs in which such questions can be asked.

Acknowledgements. We are grateful to Shagnik Das and Nina Kamčev for helpful conversations and are particularly indebted to Shagnik for writing up an early draft of Section 3. We would also like to thank the anonymous referees for their detailed and insightful reports.