Interference Effects in $t{\bar t}$ Production at the LHC as a Window on New Physics

TL;DR

This paper investigates how interference effects between new scalar or pseudoscalar particles and QCD background in top quark pair production at the LHC can serve as a probe for new physics beyond the Standard Model.

Contribution

It provides a detailed analysis of interference effects in $t\bar t$ production across various BSM models, including singlet scalars, two-Higgs doublets, and supersymmetry, with realistic detection considerations.

Findings

Interference effects can significantly alter the $t\bar t$ signal shape.

Search strategies can potentially detect or constrain new scalar resonances.

Interference patterns depend on model parameters and can be used to distinguish BSM scenarios.

Abstract

Many extensions of the Standard Model contain (pseudo)scalar bosons with masses in the TeV range. At hadron colliders, such particles would predominantly be produced in gluon fusion and would decay into top quark pair final sates, a signal that interferes with the large QCD background $gg \to t\bar t$. This phenomenon is of interest for searches for by the LHC experiments. Here, we consider the signal and background interference in this process and study it in various benchmark scenarios, including models with extra singlet (pseudo)scalar resonances, two-Higgs doublet models, and the minimal supersymmetric extension of the SM with parameters chosen to obtain the measured light Higgs mass (the hMSSM). We allow for the possible exchanges of beyond the SM vector-like particles as well as scalar quarks. We calculate the possible interference effects including realistic estimates of the attainable detection efficiency and mass resolution. Studies of our benchmark scenarios indicate that searches with an LHC detector could permit the observation of the $t\bar t$ final states or constrain significantly large regions of the parameter spaces of the benchmark scenarios.

Tables

Table 1. Table 1: The couplings of the h , H , A ℎ 𝐻 𝐴 h,H,A bosons to fermions and gauge bosons in Type-I and -II 2HDMs relative to SM Higgs couplings; the H ± superscript 𝐻 plus-or-minus H^{\pm} couplings to fermions have factors similar to those of the CP–odd boson A 𝐴 A .

$\Phi$	${\hat{g}}_{\Phi u\overline{u}}$		${\hat{g}}_{\Phi d\overline{d}}$		${\hat{g}}_{\Phi VV}$
	Type I	Type II	Type I	Type II	Type I/II
$h$	$\cos \alpha /\sin \beta$	$\cos \alpha /\sin \beta$	$\cos \alpha /\sin \beta$	$-\sin \alpha /\cos \beta$	$\sin \left(\beta -\alpha \right)$
$H$	$\sin \alpha /\sin \beta$	$\sin \alpha /\sin \beta$	$\sin \alpha /\sin \beta$	$\cos \alpha /\cos \beta$	$\cos \left(\beta -\alpha \right)$
$A$	$\cot \beta$	$\cot \beta$	$\cot \beta$	$\tan \beta$	$\mathrm{\hspace{0.33em}0}$

Equations

$${\mathcal{A}}_{gg\to t\bar{t}}^{\Phi }=-\sum _{\Phi }\frac{{\mathcal{A}}_{gg\Phi }\hat{s}{\mathcal{A}}_{\Phi tt}}{\hat{s}-M_{\Phi }^2+i{M}_{\Phi }{\Gamma }_{\Phi }}+{\mathcal{A}}_{ggtt},$$

$${\mathcal{A}}_{gg\Phi }=\frac{{\alpha }_s}{8\pi v}\hat{s}\sum _Q{\hat{g}}_{\Phi QQ}{A}_{1/2}^{\Phi }({\tau }_Q),$$

$$A_{1/2}^H(\tau )=2\left[\tau +(\tau -1)f(\tau )\right]{\tau }^{-2},A_{1/2}^A(\tau )=2{\tau }^{-1}f(\tau ):$$

$$\displaystyle f(\tau)\ \equiv\;\left\{\begin{array}[]{ll}\displaystyle\arcsin^{2}\sqrt{\tau}&{\rm for}\;\tau\leq 1\,,\\ \displaystyle-\frac{1}{4}\left[\log\frac{1+\sqrt{1-\tau^{-1}}}{1-\sqrt{1-\tau^{-1}}}-i\pi\right]^{2}&{\rm for}\;\tau>1\,.\end{array}\right.$$

$$\frac{\mathrm{d}\hat{\sigma }}{\mathrm{d}z}=\frac{\mathrm{d}{\hat{\sigma }}_B}{\mathrm{d}z}+\frac{\mathrm{d}{\hat{\sigma }}_S}{\mathrm{d}z}+\frac{\mathrm{d}{\hat{\sigma }}_I}{\mathrm{d}z},$$

$$\frac{\mathrm{d}{\hat{\sigma }}_B}{\mathrm{d}z}$$

$$\frac{\mathrm{d}{\hat{\sigma }}_S}{\mathrm{d}z}$$

$$\frac{\mathrm{d}{\hat{\sigma }}_I}{\mathrm{d}z}$$

$$\Gamma (\Phi \to t\bar{t})\equiv {\Gamma }_{\Phi }(\hat{s})=3\frac{G_F{m}_t^2}{4\sqrt{2}\pi }{\hat{g}}_{\Phi t\bar{t}}^2{\beta }_t^{p_{\Phi }}\frac{\hat{s}}{M_{\Phi }}.$$

$${\mathcal{L}}^{\mathrm{newYukawa}}\supset -g_{Ht\bar{t}}\bar{t}tH\mathrm{or}i{g}_{At\bar{t}}\bar{t}{\gamma }_{5}tA.$$

$$g_{\Phi t\bar{t}}=\frac{m_t}{v}\times {\hat{g}}_{\Phi t\bar{t}}$$

$${\hat{g}}_{hVV}=\frac{g_{hVV}}{g_{H_{\mathrm{SM}}VV}}=\sin (\beta -\alpha ),\text{leavevmode}\text{leavevmode}\text{leavevmode}{\hat{g}}_{HVV}=\frac{g_{HVV}}{g_{H_{\mathrm{SM}}VV}}=\cos (\beta -\alpha ).$$

$${\hat{g}}_{hAZ}={\hat{g}}_{h{H}^{\pm }W}={\hat{g}}_{AVV}={\hat{g}}_{HVV}=0,\text{leavevmode}\text{leavevmode}\text{leavevmode}{\hat{g}}_{HAZ}={\hat{g}}_{H{H}^{\pm }W}={\hat{g}}_{A{H}^{\pm }W}={\hat{g}}_{hVV}=1.$$

$$\mathrm{Type}-\mathrm{I}\text{leavevmode}:\text{leavevmode}{\hat{g}}_{\Phi t\bar{t}}=\cot \beta ,{\hat{g}}_{\Phi b\bar{b}}={\hat{g}}_{\Phi \tau \tau }=\cot \beta ,$$

$$\mathrm{Type}-\mathrm{II}\text{leavevmode}:\text{leavevmode}{\hat{g}}_{\Phi t\bar{t}}=\cot \beta ,{\hat{g}}_{\Phi b\bar{b}}={\hat{g}}_{\Phi \tau \tau }=\tan \beta .$$

$$2HDM:M_h=125\text{leavevmode}\mathrm{GeV},\alpha =\beta -\frac{\pi }{2},M_H\approx M_A,M_{H^{\pm }}\ge \max (M_H,M_A),$$

$$\displaystyle\Delta{\cal M}^{2}_{22}\approx\Delta M_{h}^{2}|^{t/\tilde{t}}_{\rm 1loop}\sim\frac{3m_{t}^{4}}{2\pi^{2}v^{2}}\bigg{[}\log\frac{M_{S}^{2}}{m_{t}^{2}}+\frac{X_{t}^{2}}{M_{S}^{2}}-\frac{X_{t}^{4}}{12M_{S}^{4}}\bigg{]}\;,$$

$$\Delta {\mathcal{M}}_{22}^2=\frac{M_h^2(M_A^2+M_Z^2-M_h^2)-M_A^2{M}_Z^2{\cos }^{2}2\beta }{M_Z^2{\cos }^2\beta +M_A^2{\sin }^2\beta -M_h^2}.$$

$$M_H^2$$

$$\alpha$$

$$\frac{\mathrm{d}{\hat{\sigma }}_S}{\mathrm{d}z}$$

$$\frac{\mathrm{d}{\hat{\sigma }}_I}{\mathrm{d}z}$$

$$\frac{\mathrm{d}{\hat{\sigma }}_S}{\mathrm{d}z}$$

$$\frac{\mathrm{d}{\hat{\sigma }}_I}{\mathrm{d}z}$$

$${\mathcal{A}}_{SUSY}^H$$

$$A_0^H(\tau )=-\left[\tau -f(\tau )\right]{\tau }^{-2}.$$

$$g_{\Phi {\tilde{t}}_i{\tilde{t}}_i}=(-i)\left(2\sqrt{\sqrt{2}{G}_F}\right){\hat{g}}_{\Phi {\tilde{t}}_i{\tilde{t}}_i}.$$

$${\hat{g}}_{H{\tilde{t}}_1{\tilde{t}}_1}$$

$${\hat{g}}_{H{\tilde{t}}_2{\tilde{t}}_2}$$

$$\sin 2{\theta }_t=\frac{2m_t{X}_t}{m_{{\tilde{t}}_1}^2-m_{{\tilde{t}}_2}^2},\cos 2{\theta }_t=\frac{m_{{\tilde{t}}_L}^2-m_{{\tilde{t}}_R}^2}{m_{{\tilde{t}}_1}^2-m_{{\tilde{t}}_2}^2}.$$

$$Y_t$$

$${\hat{g}}_{H{\tilde{t}}_1{\tilde{t}}_1}$$

$$Y_t^{\mathrm{light}\text{leavevmode}\mathrm{stop}}$$

$$\frac{\mathrm{d}\sigma }{\mathrm{d}{m}_{t\bar{t}}}=2\sqrt{\hat{s}}F(\hat{s},s)\cdot \hat{\sigma }(\hat{s}),$$

Full text

1 Introduction

Searches for new physics at the CERN LHC exploit a variety of different signatures, including searches for resonant enhancements in various final states, searches for non-resonant excesses in the distributions of one, two or more particles, and events with missing transverse momentum. All of these signatures have contributed to the discovery and characterization of the Higgs boson [1, 2], and are being pursued in ongoing searches for new physics such as new $Z^{\prime}$ or $W^{\prime}$ gauge bosons [3, 4, 5, 6, 7] or supersymmetry [8, 9]. Many authors have pointed out that interference effects in the spectra of specific particle pairs are of particular interest for the discovery and exploration of spin-0 states. One example is the constraint that off-shell interference in $ZZ$ final states imposes on the total width of the observed Higgs boson [10, 11] and many authors have also emphasized the potential utility of interference effects in $\gamma\gamma$ [12, 13, 14] and $t{\bar{t}}$ [15, 16, 17, 18, 19] final states for constraining the properties of heavier spin-0 bosons. Some of these studies were stimulated by the ill-fated hint of a 750-GeV feature in the $\gamma\gamma$ spectrum at the LHC [20, 21], but their principles have broader applicability.

There are many scenarios for physics beyond the Standard Model (BSM) that feature additional scalar or pseudoscalar bosons with masses in the few hundred to the TeV range that could be accessible to experiments at the LHC. The simplest of these models contain an additional SM-singlet (pseudo)scalar boson, and many others postulate a second SU(2)-doublet of Higgs bosons (two–Higgs doublet models or 2HDMs) [22]. Prominent among the latter is the minimal supersymmetric extension of the SM (the MSSM) [23], whose parameters may be constrained so that it predicts correctly the mass $\simeq 125$ GeV of the observed Higgs boson (the hMSSM) [24, 25]. The interferences with SM backgrounds that such bosons would produce in various two–body channels provide an interesting physics opportunity for the LHC experiments and a novel window on possible BSM physics.

Of particular interest are $t{\bar{t}}$ final states, for several reasons. If the sought-for spin-0 boson has a mass in the range of several hundred GeV, the decays into $t{\bar{t}}$ final states would be kinematically allowed with a branching ratio that may be large, yielding a signal-to-background ratio that is more favourable than for decays into lighter fermions, for instance. Moreover, unlike the case of a spin-1 boson, there is no good reason why the couplings of spin-0 bosons to fermions should be generation-independent. Indeed, this is known not to be the case for the SM Higgs boson, which couples to particles proportionally to their mass and, hence, has a very large $t{\bar{t}}$ coupling. This is also the case in 2HDMs such as the MSSM if the ratio, $\tan\beta$, of the two Higgs vacuum expectation values (vevs) is not very large, and the additional scalar and pseudoscalar states $\Phi=H/A$ states are relatively heavy, in which case the decay modes $\Phi\rightarrow t\bar{t}$ tend to dominate. In such a case the $gg\rightarrow H/A$ production cross sections are still substantial, thanks to the large Higgs coupling to the top quarks that mediate the production process through triangle diagrams, and may be further enhanced if there are additional vector–like quarks (VLQs) or scalar quarks (such as the stop squarks in the MSSM). Therefore, searches at the LHC for heavy scalar bosons decaying into $t{\bar{t}}$ final states are of special interest, even mandatory as the only way to probe regions of the 2HDM or MSSM parameter space with large $M_{H/A}$ and small $\tan\beta$ values.

As we show in this paper using various benchmark scenarios, including a model with an extra scalar singlet and models with two Higgs doublets such as the 2HDM and the MSSM, interference signatures in the cross section for producing $t{\bar{t}}$ final states at the LHC provide distinctive and sensitive signatures for searches of new physics beyond the SM.

Unlike the case of a spin–one electroweak resonance, a peak in the $t{\bar{t}}$ invariant mass distribution, as expected in the narrow–width approximation, is not the only possible signature of a resonance. Indeed, in the case of resonances such as heavy neutral gauge bosons $Z^{\prime}$ or bosonic Kaluza–Klein excitations, this situation does not occur: these states are mainly produced in the $q\bar{q}$ annihilation channel and the electroweak process $q\bar{q}\rightarrow V\rightarrow t\bar{t}$ does not interfere with the background from the colored $q\bar{q}\rightarrow t\bar{t}$ channel due to $s$-channel gluon exchange, leading one to expect only a peak in the invariant mass distribution in these cases 111However, in the general electroweak processes $q\bar{q}\to f\bar{f}$, with $f=e,\mu$ for instance, there is also an interference between the contribution of the spin-1 $V$ boson and the $Z,\gamma$ exchange contributions; but as the new resonance $V$ is expected to be rather heavy and narrow, the effect of the interference is entirely negligible if one restricts to invariant fermion masses close to the resonance mass [26, 27]. This interference can be important in $e^{+}e^{-}$ collisions though, in particular to detect the resonance effects below the kinematical threshold where it can be produced on–shell, see for instance Ref. [28]. Note that interference effects in the production of Kaluza-Klein gluons in $gg\!\to\!g_{KK}\!\to\!t\bar{t}$ have been discussed in e.g. Ref. [29] but they are small.. In turn, $gg\rightarrow H/A$ production followed by the $H/A\to t\bar{t}$ decay in a 2HDM for instance will interfere with the QCD background for top quark pair production that, at LHC energies, is mainly generated by the gluon–fusion process $gg\rightarrow t{\bar{t}}$. The signal-to-background interference pattern will depend on the CP nature(s) of the $\Phi=H/A$ boson(s) as well as mass(es) and total decay width(s) [15, 16]. The interference may be either constructive or destructive, varying across the resonance mass, leading to a rather complex signature exhibiting a peak-and-dip structure in the $t{\bar{t}}$ invariant mass distribution.

These issues were discussed at the theoretical level in Refs. [30, 25] in the context of the MSSM, which contains, in addition to a light CP–even $h$ boson that is identified with the 125 GeV Higgs state observed at the LHC, another CP–even or scalar particle $H$, a CP–odd or pseudoscalar particle $A$ and two charged Higgs states $H^{\pm}$. In this case, the situation is relatively simple to discuss as the Higgs sector can be described using only two input parameters, namely $\tan\beta$ and $M_{A}$, if the value $M_{h}=125$ GeV is accounted for by the large radiative corrections that occur in the Higgs sector, the so-called hMSSM approach [24, 25]. When M_{A}\raisebox{-3.69899pt}{\leavevmode\nobreak\ \shortstack{>$\\ [-0.07cm]$\sim}}\leavevmode\nobreak\ 2m_{t}, the MSSM approaches the decoupling limit where $h$ is SM–like while $M_{H}\approx M_{H^{\pm}}\approx M_{A}$ and $A/H$ have similar couplings. One can thus describe the $gg\to H/A\to t\bar{t}$ process simply in terms of the two inputs $M_{A}$ and $\tan\beta$, and hence in the $[M_{A},\tan\beta]$ parameter space, like other Higgs-sector processes in the MSSM.

However, in a general 2HDM there are in principle seven parameters: the four Higgs masses, two mixing angles $\alpha$ and $\beta$ and the mixing mass $m_{12}$, which render a complete analysis a daunting task. We therefore need a benchmark scenario suitable for experimental studies of the process and which illustrates the main important points. Here, we show that one can describe the $t{\bar{t}}$ search in some generality by:

• •

assuming the alignment limit $\alpha\simeq\beta-\frac{\pi}{2}$, which renders the lighter $h$ boson SM–like as favoured by LHC data [31, 32], which suppresses many decay channels that might otherwise compete with the $H/A\rightarrow t\bar{t}$ mode, such as $H\rightarrow WW,ZZ,hh$ or $A\rightarrow hZ$, and makes gluon fusion (together with associated production with $t,b$ quarks) the only $H/A$ production mode;

• •

assuming a small difference between the $H$ and $A$ masses, |M_{H}-M_{A}|\raisebox{-3.69899pt}{\leavevmode\nobreak\ \shortstack{<$\\ [-0.07cm]$\sim}}\leavevmode\nobreak\ M_{Z} in most cases, so as to be consistent with the electroweak precision data (particularly the constraint from the $\rho$ parameter) [33, 34], thereby suppressing many other decay channels that might compete with $H/A\rightarrow t{\bar{t}}$, such as $H\rightarrow AZ$ or $A\rightarrow HZ$; we also assume that $H^{\pm}$ is heavier than max$(M_{A},M_{H})$ in order to suppress decays like $H/A\rightarrow WH^{+}$;

• •

finally, ignoring the mass mixing parameter $m_{12}$, as it enters only in the Higgs self-couplings that do not affect our discussion here.

This benchmark scenario is almost as simple as the hMSSM, and allows us to study the process $pp\rightarrow H/A\rightarrow t{\bar{t}}$ also in the $[M_{A},\tan\beta]$ plane. In this initial exploration, in addition to the assumptions above, we restrict our attention to the range M_{A,H}\raisebox{-3.69899pt}{\leavevmode\nobreak\ \shortstack{>$\\ [-0.07cm]$\sim}}\leavevmode\nobreak\ 350 GeV in which the two–body $A/H\to t\bar{t}$ is kinematically allowed 222The three-body decay $H/A\to tt^{*}\to tbW$ below the kinematic threshold is in general suppressed, but can be significant for values of $\tan\beta$ close and below unity; see for instance Ref. [35]. and to values $\frac{1}{2}\leq\tan\beta\leq 5$–7 for which the $\Phi t\bar{t}$ couplings are large and, hence, the $t\bar{t}$ branching fractions are significant.

As well as the case of the SM but with the addition of an isosinglet scalar or pseudoscalar boson that interacts with the top quark with a coupling simply $\propto m_{t}/v$, we also consider the possibility of an enhancement of the $gg\to H/A$ triangular loop contribution by additional matter particles. We will take two examples for illustration: $i)$ a vector–like quark (VLQ) with a mass in the TeV range and general couplings to the (pseudo)scalar resonance, and $ii)$ the scalar top squark in the MSSM, which can be relatively light and has large couplings to the CP–even $H$ boson. The shape and magnitude of the interference between signal and backgrounds can be significantly modified in these cases.

The rest of the paper is organised as follow. In the next Section, we describe and give the analytical forms at leading order of the pure signal, pure background and interference contribution in the $gg\to t\bar{t}$ process. In Section 3, we introduce the three benchmark scenarios that we will consider, the isosinglet resonance, the 2HDM and MSSM cases, and the models in which extra contributions from VLQs or scalar top squarks are present. In Section 4, we present the method of computation of the expected sensitivity and exclusion potential for the $t\bar{t}$ signal. This analysis targets the $\ell+\text{jets}$ final state and exploits the $m_{{t\bar{t}}}$ distribution smeared to emulate the detector resolution. In Section 5, we present the potential sensitivities in all the benchmark scenarios that we consider. A brief summary is given in Section 6.

2 The $\mathbf{gg\rightarrow H/A\rightarrow t\bar{t}}$ process at the LHC

The discussion in the Introduction suggests that the gluon fusion production processes $gg\rightarrow\Phi=H/A$, followed by decays of the Higgs bosons into top quark pairs, $H/A\to t\bar{t}$, are promising channels for exploring the parameter spaces of 2HDMs such as the MSSM. This is the case particularly for low $\tan\beta$ values: the $\Phi t\bar{t}$ couplings are only slightly suppressed for \tan\beta\raisebox{-3.69899pt}{\leavevmode\nobreak\ \shortstack{<$\\ [-0.07cm]$\sim}}\leavevmode\nobreak\ 3–5 and enhanced for \tan\beta\raisebox{-3.69899pt}{\leavevmode\nobreak\ \shortstack{<$\\ [-0.07cm]$\sim}}\leavevmode\nobreak\ 1, compared to those in the SM, and the $\Phi\rightarrow t\bar{t}$ decay modes are by far the dominant ones. The search for a peak in the invariant mass distribution of the top quark pairs produced at the LHC, which would signal the presence of a resonance that couples to $t\bar{t}$ states, is thus a priority. It would complement the search for resonances decaying into $\tau\tau$ pairs that, as is well known, are more sensitive to the high- and moderate-$\tan\beta$ regions of the MSSM. However, a peak in the invariant mass of the $t\bar{t}$ system, which one generally expects in the narrow–width approximation that is valid in the case of the models we study, as the $H/A$ total decay widths are in general small, is not the only possible signature of a Higgs resonance.

This is because the amplitude for the Higgs signal due to the gluon fusion process $gg\rightarrow\Phi$ interferes with the QCD amplitude for the pair-production of top quarks, which is mediated by the $q\bar{q}\rightarrow t\bar{t}$ annihilation and $gg\rightarrow t\bar{t}$ fusion channels, the latter being by far the dominant component at the LHC. The two Feynman diagrams for the QCD $gg\rightarrow t\bar{t}$ process at leading order are shown in Fig. 1, together with the Feynman diagram for the resonant signal. The interference depends on the CP nature of the $\Phi$ boson as well as on its mass and total width, and may be either constructive or destructive, leading to a rather complex signature with a peak–dip structure in the $t\bar{t}$ invariant mass distribution, as already mentioned.

This feature has been known for some time, and has in particular been discussed in the context of a heavy SM Higgs boson [15, 16, 17] and, by extension, also for the case of a CP–even Higgs boson in a 2HDM [19]. However, in this case the situation is more complicated, as there are two possible resonances with different CP quantum numbers and hence two possible peaks if the interference is positive and/or dips if the interference is negative, which would be close together if the $H$ and $A$ states are nearly degenerate. Compared to the SM Higgs case, this situation has been relatively rarely discussed in the literature, though see for instance Refs. [36, 37, 38, 39, 40] and the detailed analyses of Refs. [18, 41], in which many resonances including CP–even and CP–odd scalar particles have been studied. In the following we summarize briefly the main aspects.

The amplitude for the $gg(\rightarrow\Phi)\rightarrow t\bar{t}$ process, including the contributions of both the resonant signal production of the state $\Phi$ with mass $M_{\Phi}$ and total decay width $\Gamma_{\Phi}$, and the continuum background, may be written as

[TABLE]

where $\hat{s}$ is the partonic centre-of-mass energy-squared and ${\cal A}_{ggtt}$ is the tree-level QCD background amplitude. The amplitude ${\cal A}_{gg\Phi}$ for resonance production $gg\rightarrow\Phi$ is induced at LO by loops of heavy quarks $Q$ and is given by [42]

[TABLE]

where the form factors $A_{1/2}^{\Phi}$ for the contributions of spin–$\frac{1}{2}$ quarks as functions of the variable $\tau_{Q}\equiv M_{\Phi}^{2}/4m_{Q}^{2}$ are given in the CP–even $H$ and CP–odd $A$ cases by

[TABLE]

with

[TABLE]

The form factors are displayed in Fig. 2 as a function of the variable $\tau$. They vanish in the zero–mass limit for the fermions, while in the infinite-mass limit they reach constant values $A_{1/2}^{H}\rightarrow\frac{4}{3}$ and $A_{1/2}^{A}\rightarrow 2$. They are real below the kinematical threshold $M_{\Phi}=2m_{Q}$ and develop imaginary parts above, reaching their maximum values just above the threshold.

The partonic differential cross section can be written in a convenient way as

[TABLE]

where $z\equiv\cos\theta$, with $\theta$ the scattering angle in the parton-parton centre-of-mass between an incoming gluon and the top quark. The various terms in eq. (7) can be written as [16]

[TABLE]

where the velocity of the final top quark in the parton-parton centre-of-mass frame is $\hat{\beta}_{t}\equiv\sqrt{1-4m_{t}^{2}/\hat{s}}$, and $p_{\Phi}=3$ (1) for the CP–even (CP–odd) scalar. With our convention for the $\Phi t\bar{t}$ coupling, the SM Higgs coupling corresponds to $\hat{g}_{ht\bar{t}}=1$. The expressions eq. (8) involve the energy-dependent heavy (pseudo)scalar partial width [42]:

[TABLE]

The total cross sections for the signal, the background and interference are then obtained by integrating the partonic cross sections over the scattering angle $\theta$ and folding them with the $gg$ luminosity function; see eq. (32).

Figure 3 shows examples of parton-level cross sections computed according to eqs. (8) but with fixed widths. These plots illustrate the richness of possible effects due to interference. The top left plot shows the parton-level cross section for the sum of the signal process and the interference for different masses of the extra singlet scalar (dashed line) or pseudo-scalar (full line). The top right plot shows the parton-level cross section for the sum of the signal process and the interference for a given mass of $500$ GeV for the extra singlet scalar (dashed line) or pseudo-scalar (full line) but for different values of its width. The lower left plot shows the dependence on the value of the $\hat{g}_{\Phi tt}$ coupling, with the cross sections scaled by additional factors for convenience, as specified in the legend. Examples of the effects of the smearing of peak and dip effects due to the mass resolution will be shown later. Other models to be discussed later exhibit analogous features.

In the case where the quark mass is such that $2m_{Q}>\sqrt{\hat{s}}$, the amplitude $A_{1/2}^{\Phi}(\tau)$ (or the function $f(\tau)$ defined earlier) is real, and the interference given by the last term in eq. (8) vanishes at $\hat{s}=M_{\Phi}^{2}$. On the other hand, for $2m_{Q}<\sqrt{\hat{s}}$, which is always the case of the bottom–quark loop and the top–quark loop above the $M_{\Phi}=2m_{t}$ threshold, the form factor $A_{1/2}^{\Phi}(\tau)$ develops an imaginary part that leads to a non–zero interference term when $\hat{s}=M_{\Phi}^{2}$. The magnitude of this interference is controlled essentially by the total decay width of the $\Phi$ boson, $\Gamma_{\Phi}$, and is larger for smaller value of $\Gamma_{\Phi}$. In the cases that we consider here, i.e., for $\tan\beta$ not much smaller than unity, $\Gamma_{\Phi}/M_{\Phi}$ is indeed small, namely \raisebox{-3.69899pt}{\leavevmode\nobreak\ \shortstack{<$\\ [-0.07cm]$\sim}}\leavevmode\nobreak\ 5% 333This is in contrast to the case of a heavy SM–like Higgs boson, which would have had a large total decay width, ${\cal O}$(500 GeV) for a mass $\sim 1$ TeV [43, 44]. This would have made the interference between signal and background irrelevant in that case..

Since the interference term may be large, the narrow–width approximation with a signal characterized by an excess in $\sigma(gg\rightarrow\Phi)\times{\rm BR}(\Phi\rightarrow t\bar{t})$ on top of the QCD background is not adequate in general, and one would observe a more complicated structure in the $t\bar{t}$ invariant-mass distribution. Depending on the Higgs masses and couplings, a peak and/or dip structure should be present, as seen in Fig. 3. In the following discussion we quantify the sensitivity to the Higgs signal in the process $pp\rightarrow\Phi\rightarrow t\bar{t}$ with $\Phi=H/A$ including the interference terms.

All the amplitudes quoted above were given at leading order (LO) in perturbation theory and important higher-order effects need to be included. The QCD corrections to the $gg\rightarrow\Phi$ production cross section, beyond the $M_{\Phi}=2m_{t}$ threshold that is relevant here, have been calculated only at NLO [45] and lead to a $K$–factor of about 1.6. It turns out, however, that the result in the heavy top quark limit is a good approximation at this order [46, 47, 48] (at least for the real part). In this approximation, the corrections have been calculated at NNLO in the scalar and pseudoscalar cases [49, 50, 51, 52] (and in the former case even at N3LO [53]). They have been implemented in recent years in the code SusHi [54, 55] and lead to a $K$-factor of about 2 in total. The QCD corrections to the decay $\Phi\rightarrow t\bar{t}$ are known to NNLO in QCD [56] and increase the LO partial width by approximately a factor $\sim 1.5$. In the case of the $pp\rightarrow t\bar{t}$ QCD background process, the QCD corrections at NLO have been known for a long time, and the resulting $K$–factor is $K_{\rm NLO}^{\rm QCD}\approx 1.3$ [57, 58]. The NNLO QCD corrections to the process have also been completed recently [59] and they increase the total cross section slightly beyond the NLO value. The electroweak corrections are rather small in both the signal and background processes, and can be ignored to first approximation.

The NLO QCD corrections to the interference between signal and background in resonant the process $gg\to H/A\to t\bar{t}$ have been calculated recently using an effective field theory approach [60]. In our case, we take into account the QCD corrections simply by rescaling the Higgs signal and the QCD background cross sections by the respective NNLO $K$-factors $K^{\rm S}_{\rm NNLO}$ (computed using the program SusHi) and $K^{\rm QCD}_{\rm NNLO}$. We then rescale the interference term by the geometrical average of the signal and QCD background NNLO $K$–factors [19], namely $K^{I}_{\rm NNLO}=\sqrt{K^{\rm QCD}_{\rm NNLO}\times K^{S}_{\rm NNLO}}$ (see Section 4 for more details about the analysis).

3 Benchmark models

3.1 Extended Higgs sectors

3.1.1 The SM with an extra singlet (pseudo)scalar

We consider first a minimal benchmark model, namely the SM supplemented by just one of the following terms for the interaction of a heavy scalar $H$ or pseudoscalar $A$ with the top quark:

[TABLE]

In a complete model, the field $H$ or $A$ should be integrated into an electroweak doublet, or the interactions should be generated via a dimension-5 (or greater) interaction including an SM Higgs doublet, but we ignore such complications here, for the purposes of illustration.

We use the SM–like Higgs coupling to fermions as a reference, expressing these new scalar and pseudoscalar couplings to the top quark in the form:

[TABLE]

where $\Phi=H$ or $A$, and with the vev $v=1/\sqrt{\sqrt{2}G_{F}}=246$ GeV. With this definition, the SM Higgs coupling $g_{ht\bar{t}}={m_{t}}/{v}$ corresponds to $\hat{g}_{ht\bar{t}}=1$. As usual, the Feynman rules associated with these couplings are obtained by multiplying $g_{\Phi t\bar{t}}$ by $(-i)$. These new interactions induce $gg\Phi$ couplings via quantum corrections.

3.1.2 Two Higgs doublet models

An excellent benchmark for studying extended Higgs sectors with a richer phenomenology is a model in which two Higgs doublets $\Phi_{1}$ and $\Phi_{2}$ break the electroweak symmetry (for a review on 2HDMs, see Ref. [22], for example). It leads to five physical states: two CP–even neutral bosons, $h$ and $H$, a CP–odd boson, $A$, and two charged $H^{\pm}$ bosons. In the general case, the masses $M_{h},M_{H},M_{A}$ and $M_{H^{\pm}}$ are free parameters, and one assumes that $h$ is the observed Higgs boson with mass $M_{h}=125$ GeV. At least two additional mixing parameters $\beta$ and $\alpha$ are needed to characterize fully the model: $\tan\beta=v_{2}/v_{1}$ is the ratio of the vacuum expectation values of the two fields with $v_{1}^{2}\!+\!v_{2}^{2}\!=\!v^{2}\!=\!{\rm(246\leavevmode\nobreak\ GeV)^{2}}$, while $\alpha$ diagonalises the CP–even $h$ and $H$ mass matrix. In a general 2HDM, there is also an additional mass parameter linking the $\Phi_{1},\Phi_{2}$ fields that enters only in the quartic couplings among Higgs bosons, and can safely be ignored in our present discussion.

The couplings of the neutral $h,H,A$ bosons to massive gauge bosons and to fermions, normalised to those of the SM Higgs boson, are given in Table 1. There is no coupling of the CP–odd boson $A$ to the vector bosons $V=W,Z$ when CP is conserved, as we consider here, but the CP–even $h$ and $H$ states share the couplings of the SM Higgs particle to vector boson pairs $VV$:

[TABLE]

Taking into account the fact that the couplings of the $h$ boson have been measured at the LHC, and found to be SM–like within 10% accuracy [61, 62], we infer that \hat{g}^{2}_{hVV}\raisebox{-3.69899pt}{\leavevmode\nobreak\ \shortstack{>$\\ [-0.07cm]$\sim}}\leavevmode\nobreak\ 0.9 and hence \cos^{2}(\beta-\alpha)\raisebox{-3.69899pt}{\leavevmode\nobreak\ \shortstack{<$\\ [-0.07cm]$\sim}}\leavevmode\nobreak\ 0.1. This constraint can be accommodated naturally in a 2HDM by invoking the alignment limit [63] in which one has $\cos(\beta-\alpha)=0$ exactly, and hence $\alpha=\beta-\frac{\pi}{2}$. We adopt the alignment limit throughout this paper, which leads to a simplified picture in which the couplings between Higgs and gauge bosons, normalised to the SM as above, become simply

[TABLE]

In contrast, the Higgs interactions with fermions are model–dependent in a 2HDM, and there are two model options commonly discussed in the literature [22]: Type–II, in which one field generates the masses of isospin down–type fermions and the other the masses of up–type quarks, and Type–I, in which one field generates the masses of all fermions. However, there is a simplification in the alignment limit $\alpha=\beta-\frac{\pi}{2}$, as the $h$ couplings to all fermions are SM–like, $\hat{g}_{hf\bar{f}}=1$, while the $H$ and $A$ couplings to a given fermion species are exactly the same, $\hat{g}_{Hf\bar{f}}=\hat{g}_{Af\bar{f}}$. Thus, one has for the $\Phi=H,A$ couplings to third-generation fermions:

[TABLE]

At high $\tan\beta$ values, \tan\beta\raisebox{-3.69899pt}{\leavevmode\nobreak\ \shortstack{>$\\ [-0.07cm]$\sim}}\leavevmode\nobreak\ 10, all couplings are suppressed in Type–I models, whereas in Type–II models the $\Phi b\bar{b}$ and $\Phi\tau\tau$ couplings are instead enhanced. At low $\tan\beta$ values, \tan\beta\raisebox{-3.69899pt}{\leavevmode\nobreak\ \shortstack{<$\\ [-0.07cm]$\sim}}\leavevmode\nobreak\ 5, the $\Phi t\bar{t}$ couplings are not suppressed and the $\Phi b\bar{b}$ and $\Phi\tau\tau$ couplings are not strongly enhanced in Type–II models. Since $m_{t}\gg m_{b},m_{\tau}$, the $\Phi t\bar{t}$ couplings are then much larger than all other fermion couplings. This is a second important simplification: at low $\tan\beta$, all heavy Higgs couplings to fermions except those to the top quarks can be ignored and, looking at eq. (15), one sees that there is no difference between the two types of 2HDM in this case 444In fact, this similarity between models extends to two other possibilities that are also discussed, namely Type–III and Type–IV models [22], which differ from Type–I and II scenarios only in the couplings of the tau lepton, which can be ignored at low $\tan\beta$ values, for our purposes..

Note that, contrary to the supersymmetric case to be discussed later where one generally assumes 1\leq\tan\beta\raisebox{-3.69899pt}{\leavevmode\nobreak\ \shortstack{<$\\ [-0.07cm]$\sim}}\leavevmode\nobreak\ 50, values of $\tan\beta$ smaller than unity are in principle possible in a general 2HDM. However, for the top quark Yukawa coupling $\propto m_{t}/\tan\beta$ to remain perturbative, one needs to impose the bound \tan\beta\raisebox{-3.69899pt}{\leavevmode\nobreak\ \shortstack{>$\\ [-0.07cm]$\sim}}\leavevmode\nobreak\ 1/3.

Another important difference between the MSSM and the general 2HDM is that, whereas in the former case one has in most cases $M_{H}\sim M_{A}\sim M_{H^{\pm}}$, in the latter case the masses $M_{A},M_{H}$ and $M_{H^{\pm}}$ are still free parameters and can, in principle, be widely different. However, high–precision data, especially the fact that the $\rho$ parameter must be very close to unity, constrain the mass splittings between some of these states. For instance, for a given $M_{A}$ value and independently of $\tan\beta$, one finds after imposing the constraints from electroweak data that one of the two masses $M_{H}$ or $M_{H^{\pm}}$ must be almost degenerate with $M_{A}$ (within $\sim 10$%), while the other mass can be widely different; see Fig. 11 of Ref. [34].

In our numerical analyses, since our main concern is the study of the $H/A\rightarrow t\bar{t}$ process, we assume that the neutral states $H$ and $A$ are almost degenerate in mass, $|M_{H}/M_{A}-1|<0.2$ (but for completeness, we have also included results for larger mass splitting up to $M_{H}-M_{A}=200$ GeV), whereas the charged Higgs boson is somewhat heavier: M_{H^{\pm}}\raisebox{-3.69899pt}{\leavevmode\nobreak\ \shortstack{>$\\ [-0.07cm]$\sim}}\leavevmode\nobreak\ M_{A},M_{H}, and hence does not affect the phenomenology of $H$ and $A$ states in the present context.

To summarize, we propose in this paper a simple 2HDM benchmark in which the processes $pp\!\rightarrow\!\Phi\!\rightarrow\!t\bar{t}$ can be studied at the LHC, while retaining the main characteristic model features:

[TABLE]

in which the $h$ state has a mass of 125 GeV and SM–like couplings as favoured by LHC Higgs data, and the relation between the $A,H,H^{\pm}$ masses satisfies the constraints from electroweak precision data, which do not allow Higgs to Higgs plus gauge boson decays to occur. The alignment limit and $M_{H}=M_{A}(1\pm 10)\%$ and $M_{H^{\pm}}\geq{\rm max}(M_{H},M_{A})$ assumptions make that all the non–fermionic decay channels can be ignored. One can thus concentrate on the $\Phi\rightarrow t\bar{t}$ decays with branching ratios close to unity in the interesting range $1/3\leq\tan\beta\leq 5$.

3.1.3 The (h)MSSM

A widely studied incarnation of the 2HDM scenario is the MSSM, which is essentially a 2HDM of Type II in which supersymmetry imposes strong constraints on the Higgs sector, so that only two parameters are independent at tree level, namely $M_{A}$ and $\tan\beta$. However, when the radiative corrections in the Higgs sector are included, in particular the dominant loop contributions from the top and stop quarks that have strong couplings to the Higgs bosons, many additional supersymmetric parameters will enter the parameterization. This is, for instance, the case of the supersymmetry-breaking scale, chosen to be the geometric average of the two stop masses $M_{S}=\sqrt{m_{\tilde{t}_{1}}m_{\tilde{t}_{2}}}$, the stop trilinear couplings $A_{t}$ and the higgsino mass $\mu$; the corrections due to other supersymmetric parameters are much smaller.

These radiative corrections are very important in particular in the CP–even neutral Higgs sector as they can shift the lightest $h$ mass from the tree–level value $M_{h}\leq M_{Z}\cos 2\beta\leq M_{Z}$ to the one $M_{h}=125$ GeV that has been measured at the LHC. The neutral CP–even Higgs masses mix via an angle $\alpha$ that diagonalises the mass eigenstates, leading to $H=\Phi_{1}^{0}\cos\alpha+\Phi_{2}^{0}\sin\alpha$ and $h=-\Phi_{1}^{0}\sin\alpha+\Phi_{2}^{0}\cos\alpha$, where $\Phi_{1,2}^{0}$ denote the neutral CP–even components of the physical Higgs fields $\Phi_{1},\Phi_{2}$ in the current-eigenstate basis. The radiative corrections are captured by a general $2\times 2$ matrix $\Delta{\cal M}_{ij}^{2}$ in which only the $\Delta{\cal M}^{2}_{22}$ entry is relevant in most cases (in particular if the $\mu$ parameter is small). It involves the stop–top sector correction that dominates by far [64, 65, 66, 67]:

[TABLE]

where $M_{S}$ is the supersymmetry-breaking scale and $X_{t}=A_{t}-\mu/\tan\beta$ is the stop mixing parameter. It has been advocated [24, 68] that, in this case, one can simply trade $\Delta{\cal M}^{2}_{22}$ for the known $M_{h}$ value using

[TABLE]

One can then simply write $M_{H}$ and $\alpha$ in terms of $M_{A},\tan\beta$ and $M_{h}$:

[TABLE]

In the case of the $H^{\pm}$ masses, the radiative corrections are small at large $M_{A}$, and one simply has $M_{H^{\pm}}\simeq\sqrt{M_{A}^{2}+M_{W}^{2}}$ to a good approximation.

This is the hMSSM approach which has been shown to provide a very good approximation to the MSSM Higgs sector [24]. An important property of the MSSM is the decoupling limit that occurs for $M_{A}\!\gg\!M_{Z}$, but it is in practice reached already for M_{A}\raisebox{-3.69899pt}{\leavevmode\nobreak\ \shortstack{>$\\ [-0.07cm]$\sim}}\leavevmode\nobreak\ 350 GeV for any valueof $\tan\beta$. In this limit, one automatically has $\alpha\rightarrow\beta-\frac{\pi}{2}$ for the CP–even mixing angle, i.e., exactly as in the alignment limit of the 2HDM discussed above. The 125–GeV $h$ state then has SM–like Higgs couplings: $\hat{g}_{hVV}=\hat{g}_{hff}=1$, while the couplings of the heavier $\Phi=H/A$ states to gauge bosons vanish: $\hat{g}_{\Phi VV}=0$, and those to fermions depend only on $\tan\beta$ and are given by eq. (15).

The other important property of the MSSM Higgs sector decoupling limit is that, besides decoupling from the massive gauge bosons, the heavy CP–odd $A$, CP–even $H$ and the charged $H^{\pm}$ bosons become almost degenerate in mass: $M_{H}\!\approx\!M_{H^{\pm}}\!\approx\!M_{A}$. This is, in fact, the only difference between the MSSM close to the decoupling limit and the 2HDM close to the alignment limit: in the latter case, the masses $M_{A},M_{H}$ and $M_{H^{\pm}}$ are free parameters and could be very different. However, for $H/A$, this is not the case in our 2HDM benchmark scenario, as we have chosen $M_{H}\approx M_{A}$, which makes the situation similar to the MSSM. Note however that in the MSSM, we will assume \tan\beta\raisebox{-3.69899pt}{\leavevmode\nobreak\ \shortstack{>$\\ [-0.07cm]$\sim}}\leavevmode\nobreak\ 1, contrary to the 2HDM case 555An advantage of the hMSSM approach is that it allows one to describe the low $\tan\beta$ region of the MSSM that has been overlooked because, for SUSY scales of order 1 TeV, values $\tan\beta<3$ were excluded because they led to an $h$ mass that is smaller than 125 GeV. The price to pay is that, for such low $\tan\beta$ values, one has to assume that the supersymmetry-breaking scale $M_{S}\gg 1$ TeV, and hence that the model is fine-tuned. Moreover, care has to be taken not to enter regimes for small values of $\tan\beta$ that cannot be accommodated with the MSSM as pointed out in Refs. [25, 68]..

3.2 Additional matter in the $gg\rightarrow\Phi$ loop

3.2.1 Additional vector-like quark contributions to $gg\rightarrow\Phi$

We consider later the possibility that additional vector-like quarks (VLQs) may contribute to the triangular loop diagram for $gg\rightarrow\Phi$. In this case, the first line in eq. (8) for the background is unchanged, but the second and third lines are modified as follows:

[TABLE]

where the sum over the quarks $Q$ includes the top quark contribution (among the Standard Model quarks) as well as the additional VLQs.

For simplicity, we will assume in our simulations that all the VLQs have the same mass as the constraints from precision electroweak data would require this to be the case if they are in an electroweak doublet. Under this assumption, the effect of the VLQs can be parameterized by their mass and the number of VLQ species, $N_{Q}$.

3.2.2 Stop squark contributions to $gg\rightarrow\Phi$

In a supersymmetric theory the effective $gg\rightarrow\Phi$ coupling may also receive significant contributions from top quark superpartners [23, 69, 70, 71, 72, 73]. In this case the second and third lines in eq. (8) are modified as follows:

[TABLE]

where [69, 23]:

[TABLE]

The form factors $A_{0}^{\Phi}$ for the contributions of spin–[math] particles as functions of the variable $\tau_{\tilde{t}_{i}}\equiv M_{H}^{2}/4m_{\tilde{t}_{i}}^{2}$, using the function $f$ defined in eq. (6), is given in the CP–even $H$ case by

[TABLE]

The relevant Feynman rules defining the couplings of the neutral heavy Higgs bosons $A,H$ of the MSSM to the stops, $g_{\Phi\tilde{t}_{i}\tilde{t}_{i}}$, can be written in the form

[TABLE]

The CP–odd Higgs boson has vanishing coupling to identical stop pairs. In the limit $M_{A}\gg M_{Z}$, the heavier CP–even Higgs boson couples to same stop pairs (which contribute to the $gg\rightarrow H$ coupling) through the following simple expressions:

[TABLE]

The squark mixing angle, $\theta_{t}$, is defined by

[TABLE]

The stop sector can be parametrized by the three inputs $m_{\tilde{t}_{L}}$ and $m_{\tilde{t}_{R}}$, the soft SUSY breaking stop masses, and $X_{t}=A_{t}-\mu/\tan\beta$, the stop mixing parameter. The physical stop masses are denoted by $m_{\tilde{t}_{1}}$ and $m_{\tilde{t}_{2}}$. The above couplings also involve the parameter $Y_{t}$

[TABLE]

For sufficiently large $Y_{t}$ values, the coupling of the heavier CP–even Higgs $H$ to stops are strongly enhanced and can be larger than its coupling to the top quark. In the limit $M_{A}\gg M_{Z}$, the couplings eq. (26) of the heavier CP–odd Higgs boson to the stops simplify to:

[TABLE]

In the following, we consider the so called “light-stop” MSSM benchmark scenario which is promoted by the LHC Higgs Cross Section Working group; see Refs.[74, 75, 76]. The benefit of this benchmark is to accommodate the constraints on the SM Higgs mass while still allowing light stops within the TeV regime. In this scenario, one fixes the SUSY parameters $M_{1}=340$ GeV, $M_{2}=\mu=400$ GeV and the stop mixing parameter $X_{t}=1$ TeV. The trilinear couplings are adjusted such that $A_{t}=A_{b}=A_{\tau}=X_{t}+\mu/\tan\beta$. The parameter $Y_{t}$ in this scenario is then

[TABLE]

In the scan we performed in the $[M_{A},\tan\beta$] plane in our analysis, the lighter stop has a mass around $324$ GeV and the heavier stop a mass around $671$ GeV.

3.3 Phenomenology at the LHC

We turn now to the phenomenology of these Higgs sectors and, in particular, that of the heavier neutral Higgs states, since the phenomenology of the lightest $h$ boson is, by construction, essentially that of a 125–GeV SM Higgs boson, which is consistent with measurements performed so far at the LHC.

First, the phenomenology of the $H/A$ bosons in the case of the SM with an extra singlet (pseudo)scalar is simple, when neglecting the $H/A$ couplings to fermions other than the heavy top quark and ignoring the coupling of the scalar $H$ boson to massive gauge bosons as well as to light $h$ bosons (the pseudoscalar or CP–odd $A$ does not couple to these states at tree–level), as would be the case in the other models that we consider later, as a consequence of either decoupling or alignment. The only way to produce such $H/A$ bosons at the LHC would be through the $gg\to H/A$ mechanism, which proceeds mainly via top quark triangle diagrams and, to a much lesser extent, in associated production with top quark pairs, $pp\to t\bar{t}H/A$, which has a cross section that is at least two order of magnitude lower than gluon-fusion. Once produced, the $H/A$ bosons decay mainly into top quark pairs, as the decays into two gluons, two photons or a photon and a $Z$ boson (again through loops of top quarks, as well as $W^{\pm}$ loops) will have extremely small rates especially in the later two cases. Hence one would have a decay branching ratio BR($\Phi\to t\bar{t})\simeq 1$ and the total decay widths of the two bosons would be almost identical to the $t\bar{t}$ partial decay widths. Fig. 4 displays the $13$ TeV production cross sections and the $\Gamma(\Phi\to t\bar{t})$ decay widths for both the scalar and pseudoscalar in the case of the SM supplemented by an isosinglet (pseudoscalar) Higgs boson, as a function of the boson mass and their reduced coupling to the top quark, $\hat{g}_{\Phi t\bar{t}}$.

The phenomenology of the Higgs sector of the MSSM has been discussed at length in the past, and we refer to Ref. [25] for a recent account. In the high–$\tan\beta$ regime, \tan\beta\raisebox{-3.69899pt}{\leavevmode\nobreak\ \shortstack{>$\\ [-0.07cm]$\sim}}\leavevmode\nobreak\ 10, the situation for the $H/A$ states is simple: they are mainly produced via the $gg$ and $b\bar{b}$ fusion processes and decay into either $b\bar{b}$ $(\approx 90\%)$ or $\tau^{+}\tau^{-}$ $(\approx 10\%)$ final states. The searches for $\tau\tau$ resonances at high invariant masses performed by ATLAS and CMS set strong constraints on the parameter space and, for instance, for $M_{A}\!\approx\!M_{H}\approx\!750$ GeV, values of \tan\beta\raisebox{-3.69899pt}{\leavevmode\nobreak\ \shortstack{>$\\ [-0.07cm]$\sim}}\leavevmode\nobreak\ 10 are excluded [3, 77]. At small and intermediate values of \tan\beta\raisebox{-3.69899pt}{\leavevmode\nobreak\ \shortstack{<$\\ [-0.07cm]$\sim}}\leavevmode\nobreak\ 10 and for $H/A$ masses below the $t\bar{t}$ threshold, one is not yet in the decoupling regime and there is a plethora of interesting channels to be considered in the search for the neutral $\Phi$ bosons, e.g., $H\rightarrow WW,ZZ,hh$ and $A\rightarrow hZ$ [25]. Some of these channels have been studied by the LHC collaborations, and some exclusion limits have been set in the hMSSM [31, 32, 78]. However, the low–$\tan\beta$ region \tan\beta\raisebox{-3.69899pt}{\leavevmode\nobreak\ \shortstack{<$\\ [-0.07cm]$\sim}}\leavevmode\nobreak\ 5 with $H/A$ masses above the $t\bar{t}$ threshold has not been probed experimentally 666The ATLAS collaboration has also published a search for $\Phi\to{t\bar{t}}$ [79] that was performed with 20.3 /fb at a centre-of-mass energy $\sqrt{s}=8$ TeV. The results were interpreted in the context of a Type-II two–Higgs doublet model where either $A$ or $H$ is very heavy, or both are degenerate in mass, $M_{A}=M_{H}$. However, this search is not very sensitive to the models we study..

Nevertheless, from the phenomenological point of view, the situation in the case \tan\beta\raisebox{-3.69899pt}{\leavevmode\nobreak\ \shortstack{<$\\ [-0.07cm]$\sim}}\leavevmode\nobreak\ 5 and M_{\Phi}\raisebox{-3.69899pt}{\leavevmode\nobreak\ \shortstack{>$\\ [-0.07cm]$\sim}}\leavevmode\nobreak\ 350 GeV, is also rather simple, and can be summarized by noting the following points [25]:

• •

As the only significant Higgs coupling is the Yukawa coupling to top quarks, $\propto m_{t}/(v\tan\beta)$, the only relevant decay mode of the $\Phi=H/A$ states is into $t\bar{t}$ final states.

• •

The total widths, generated mostly by these decays: $\Gamma_{\Phi}\approx\Gamma(\Phi\!\rightarrow\!t\bar{t})$, scale like $M_{\Phi}/\tan^{2}\beta$ and are \raisebox{-3.69899pt}{\leavevmode\nobreak\ \shortstack{>$\\ [-0.07cm]$\sim}}\leavevmode\nobreak\ 1 GeV. For $\tan\beta\!\approx\!1$, they are ${\cal O}(5)$% of the $\Phi$ masses.

• •

As the $\Phi$ states do not couple to massive gauge bosons, and only weakly to all fermions except for tops, the main Higgs production channel is the gluon-fusion process $gg\rightarrow\Phi$, in which the top quark loop generates the dominant contribution.

The production cross sections $\sigma(gg\rightarrow\Phi)$ at the 13 TeV LHC, the branching ratio BR$(\Phi\rightarrow t\bar{t})$ and the total decay widths $\Gamma_{\Phi}$ are shown in Fig. 5 in the $[M_{A},\tan\beta]$ parameter plane for $\Phi=A$ (left) and $\Phi=H$ (right), assuming the mass $M_{H}$ and and the angle $\alpha$ as given in the hMSSM. One clearly sees that, indeed, for \tan\beta\raisebox{-3.69899pt}{\leavevmode\nobreak\ \shortstack{<$\\ [-0.07cm]$\sim}}\leavevmode\nobreak\ 3–5 and M_{\Phi}\raisebox{-3.69899pt}{\leavevmode\nobreak\ \shortstack{>$\\ [-0.07cm]$\sim}}\leavevmode\nobreak\ 400 GeV, the search channel $gg\rightarrow\Phi\rightarrow t\bar{t}$ is worth investigating at the LHC, as the production rate is significant. Note also that, in the MSSM, since one is not exactly in the decoupling limit for small $\tan\beta$ values and $M_{A}$ not very large, differences are visible between the $H/A$ plots in Fig. 5.

As discussed previously, the situation could in principle be slightly different in a 2HDM for two reasons. First, the alignment limit is not exact and, for $\cos^{2}(\beta-\alpha)\approx 0.1$, decay channels such as $A\rightarrow hZ$ and $H\rightarrow WW,ZZ,hh$ would be possible. However, for M_{\Phi}\raisebox{-3.69899pt}{\leavevmode\nobreak\ \shortstack{>$\\ [-0.07cm]$\sim}}\leavevmode\nobreak\ 400 GeV and \tan\beta\raisebox{-3.69899pt}{\leavevmode\nobreak\ \shortstack{<$\\ [-0.07cm]$\sim}}\leavevmode\nobreak\ 3, the $\Phi\rightarrow t\bar{t}$ decays are overwhelming. The branching ratios for these additional channels are very small, and can be ignored in a first approximation.

A second difference is that, as also noted previously, one does not have automatic mass degeneracy between the heavy $A,H$ and $H^{\pm}$ states in a general 2HDM, so other decay channels with unsuppressed couplings might occur, as shown by the second set of couplings in eq. (13): $A\rightarrow HZ,H^{\pm}W$ and/or $H\rightarrow AZ,AW^{\pm}$. When kinematically accessible these decays can be very important, in particular at relatively high $\tan\beta$ values, and could suppress drastically the branching ratios for $A/H\rightarrow t\bar{t}$ decays. In particular, BR($\Phi\rightarrow t\bar{t})$ drops drastically when the channels $H/A\rightarrow A/H+Z$ are kinematically open. The $H^{\pm}$ state is expected to be rather heavy as a results of constraints from heavy flavor physics [80], but its impact if $\Phi\rightarrow H^{\pm}W^{\mp}$ were kinematically open would have been exactly the same.

Nevertheless, as already mentioned, the constraints from high–precision electroweak data require two Higgs masses to be sufficiently close in mass for some of these decays to be kinematically closed at the two–body level. These constraints, added to those on the light $h$ state from LHC data, motivate our benchmark scenario, which is close to the alignment limit that suppresses most of these decay channels and $|M_{H}/M_{A}-1|\lesssim 0.1$, while $M_{H^{\pm}}\geq{\rm max}(M_{H},M_{A})$, in which case the neutral Higgs to charged Higgs decay channels are kinematically closed. In this benchmark, one can concentrate on the interesting $\Phi\rightarrow t\bar{t}$ decays with branching ratios close to unity in the interesting range $1/3\leq\tan\beta\leq 5$.

The production cross sections $\sigma(gg\rightarrow\Phi)$ at the 13 TeV LHC, the branching ratio BR$(\Phi\rightarrow t\bar{t})$ and the total decay widths $\Gamma_{\Phi}$ in our 2HDM benchmark scenarios with $M_{H}=M_{A}$ are shown in Fig. 6 in the $[M_{A},\tan\beta]$ parameter plane for $\Phi=A$ (left) and $\Phi=H$ (right). We also considered in our analysis various mass splittings between the 2HDM CP–even and CP–odd heavy Higgs bosons, namely $M_{H}-M_{A}=10,50,100,200$ GeV.

4 Simulation of experimental sensitivity

In this Section we compute the expected sensitivity and exclusion potential for each signal hypothesis, based on the distribution of the invariant mass of the ${t\bar{t}}$ system, $m_{{t\bar{t}}}$. The SM ${t\bar{t}}$ production process is by far the dominant background in the targeted $\ell+\text{jets}$ final state and the only one considered in this study. It is described with the help of a Monte–Carlo (MC) simulation. Distributions of $m_{{t\bar{t}}}$ for the BSM contribution are built via a computationally-efficient approximation that uses MC simulations of a small number of signal hypotheses to construct an economical parameterization. The statistical analysis takes into account the main systematic uncertainties affecting the $m_{{t\bar{t}}}$ distribution. All the software code used in this study is made available at [81].

We model SM $pp\rightarrow t\bar{t}$ events with decaying top quarks at leading order using MadGraph _aMC@NLO 2.6.0 [82]. Only final states with exactly one electron or muon are considered. The factorization and renormalisation scales are set to $\frac{1}{2}m_{{t\bar{t}}}$, following the choice in Ref. [19]. The parton distribution functions (PDFs) are taken from the set PDF4LHC15_nlo_30_pdfas [83], as provided in LHAPDF 6.1.6 [84]. We have generated $5\times 10^{6}$ events for a nominal mass of the top quark of 173 GeV, and the same number for each of the $\pm 0.5$ GeV variations in its mass. Showering and hadronisation for the produced events are performed using Pythia 8.230 [85] with the Monash 2013 tune [86]. Stable particles are clustered into jets using the anti-$k_{\text{T}}$ algorithm [87] with a cone size of 0.4. For convenience, the last two steps are done with the help of the Delphes 3.4.1 framework [88], which, however, is not employed to simulate the detector response. To account for higher-order corrections in SM ${t\bar{t}}$ production, the sample is normalized to the inclusive cross section computed at the $\text{NNLO}+\text{NNLL}$ precision with the program Top++ 2.0 [89]. This corresponds to applying a flat $K$-factor of 2.0.

We produced MC samples for reference signal hypotheses in the hMSSM with $M_{A}=400,500,600,700$, and 1000 GeV and various values of $\tan\beta$. A custom MadGraph model was used, which is realized as the SM with two additional neutral Higgs bosons that are pure CP eigenstates. The effective coupling to gluons was implemented following Ref. [45], with only top quarks included in the loop 777The selection efficiency parameterized as a function $m_{{t\bar{t}}}$  does not change significantly when there are extra loop contributions from VLQs or stops squarks.. The masses of the additional Higgs bosons, their total widths and couplings to top quarks are free parameters of this model, and they are fixed according to the model predictions for each point in the $[M_{A},\tan\beta]$ plane. Apart from the differences in the models, signal events are generated in the same way as for the SM ${t\bar{t}}$ events described above. Samples for the resonant part of the BSM contribution and the interference are produced independently. Owing to the destructive nature of the interference in certain regions of the phase space, some events receive negative weights, and the total cross section for the interference sample can be negative. For every value of $M_{A}$, around $10^{6}$ events have been generated for each CP eigenstate for both the resonant and interference parts.

The event selection made was dictated by the targeted $\ell+\text{jets}$ final state. The only charged lepton in the final state is required to have $p_{\text{T}}>30$ GeV and $|\eta|<2.4$. Each event must contain at least four jets with $p_{\text{T}}>20$ GeV and $|\eta|<2.4$. Jets that overlap with the lepton within $\Delta R<0.4$ are removed. Two of the jets must be matched to the $b$ quarks, thus emulating $b$-tagging. No selection on missing $p_{\text{T}}$ is applied, since the $pp\rightarrow\text{jets}$ background is not considered in this study. The combined efficiency of lepton identification and $b$-tagging, which is estimated to be 30% [90, 91, 92, 93], is included with the event weights. In order to emulate the effects of detector resolution, Gaussian smearing is applied to the parton-level invariant mass $m_{{t\bar{t}}}$ in the events selected. Two benchmark values for the $m_{{t\bar{t}}}$ resolution are considered, namely 10% and 20%.

Generating MC samples for each signal hypothesis would not be practical. Instead, $m_{{t\bar{t}}}$ distributions for the BSM contribution are constructed starting from the parton-level cross sections $\hat{\sigma}(\hat{s})$ given in Section 3 (with the $K$-factors included as described in Section 2), and the cross sections are first convoluted with the PDFs. The differential cross section in $m_{{t\bar{t}}}$ can be computed as

[TABLE]

with $s=(13\,\text{TeV})^{2}$ being the squared centre-of-mass collision energy and

[TABLE]

where $f_{g}(x)$ is the gluon PDF and $x$ is the fraction of the proton’s momentum carried by the gluon. The integration over one of the two $x$ variables has already been carried out, benefiting from the fact that the parton-level cross section depends only on $\hat{s}$. Since the function $F$ eq. (32) does not depend on the parton-level cross section, it can be precomputed, permitting a very fast convolution eq. (31).

To mimic the event selection, $\mathrm{d}\sigma/\mathrm{d}m_{{t\bar{t}}}$ is multiplied by the selection efficiency. It is computed as a function of the parton-level $m_{{t\bar{t}}}$ from the MC samples for the reference signal hypotheses. The results are shown in Fig. 7. As can be seen, the efficiencies for the resonant part and the interference differ from each other as well as from SM $gg\rightarrow{t\bar{t}}$, which is included for completeness. The signal efficiencies obtained are fitted with a cubic function of $\ln m_{{t\bar{t}}}$, which has been found empirically to provide an accurate description. These functions are also plotted in Fig. 7. The branching ratio for the targeted final state, which is $8/27$, is also included, together with the 30% efficiency of lepton identification and $b$-tagging. The resulting combined fitted selection efficiency is denoted by $\epsilon(m_{{t\bar{t}}})$ in the following.

Finally, the parton-level $m_{{t\bar{t}}}$ is smeared using a convolution with a Gaussian kernel:

[TABLE]

where $r$ is the specified $m_{{t\bar{t}}}$ resolution. The integral is computed numerically, truncating it to the segment $m_{{t\bar{t}}}\cdot(1\pm 3r)$. The resulting differential cross section is plotted in Fig. 8 for a representative hMSSM scenario and various values of the resolution. We see that the fine details of the mass distribution are lost when the smearing is 10% or more, but a peak-and-dip structure is still potentially observable.

The statistical analysis exploits the distribution of the smeared $m_{{t\bar{t}}}$ after the (emulated) event selection. The distribution is described by a histogram covering the range $350$ GeV $<m_{{t\bar{t}}}<1200$ GeV and contains 24 bins with relative widths of about 5%. The distribution for ${t\bar{t}}$ events due to the SM is constructed from the MC sample directly, whilst the distribution for the BSM component is built for each signal hypothesis by integrating $\mathrm{d}\tilde{\sigma}/\mathrm{d}m_{{t\bar{t}}}$ eq. (33) over each bin of the histogram. Both distributions are scaled to the desired integrated luminosity.

The $m_{{t\bar{t}}}$ distributions are affected by various systematic uncertainties that we estimate as follows. In the signal, the renormalisation scale in the matrix element is varied by a factor 2 in each direction, simultaneously for both the resonant part and the interference. A number of uncertainties in the SM ${t\bar{t}}$ background are considered. The overall rate is varied by 10%, which represents theoretical uncertainties in the inclusive cross section and some experimental uncertainties, such as the uncertainty in the $b$-tagging efficiency [93], that affect mostly the overall rate. Values of $m_{{t\bar{t}}}$ are rescaled as $(1\pm\alpha)\,m_{{t\bar{t}}}$, $\alpha=0.01$, which serves as a conservative proxy for the uncertainty in the jet energy scale [94]. Renormalisation and factorization scales in the matrix element are varied independently by factors of 2. Since theoretical uncertainties in the overall ${t\bar{t}}$ rate have already been covered by the corresponding uncertainty, the distributions obtained in each variation are rescaled so that the inclusive cross section does not change. The variation in the renormalisation scale is done simultaneously with its counterpart in the BSM component. The renormalisation scale used in the final-state radiation, which is controlled by the Pythia parameter TimeShower:renormMultFac, is also varied by a factor of 2, using MC samples processed with alternative Pythia configurations. The mass of the top quark is varied by $\pm 0.5$ GeV [95], using the corresponding dedicated MC samples. Finally, 30 uncertainties provided in the PDF set as well as the variation of $\alpha_{s}$ in PDF are also included. The MC statistical uncertainties are ignored, effectively assuming that they are much smaller than statistical uncertainties in the data.

Several systematic variations for the SM ${t\bar{t}}$ events (most notably, those constructed from dedicated MC samples) are affected by statistical fluctuations. The fluctuations are suppressed by smoothing the relative deviations from the nominal SM ${t\bar{t}}$ distribution using a version of the LOWESS algorithm [96, 97]. The value of the relative deviation in each bin of the distribution is replaced by the result of the weighted least-squares linear fit, in which nearby bins receive larger weights than those that are further away from the bin of interest.

The statistical analysis is implemented in the RooStats framework [98]. The model, which includes all the uncertainties described above, is constructed with the help of the HistFactory toolkit [99]. The combined BSM contribution is scaled by an auxiliary ‘signal strength’ parameter $\mu$, such that with $\mu=1$ the model describes the full $m_{{t\bar{t}}}$ distribution with the nominal BSM component, and $\mu=0$ reproduces the SM. Test statistics defined in Ref. [100], which are based on the profile likelihood ratio, are used, and their distributions are described according to the asymptotic formulae provided in the same reference. For each signal hypothesis, two kinds of results are obtained.

First, the expected significance is computed on the Asimov data set [100] with $\mu=1$. In this case the null hypothesis is $\mu=0$, the alternative is $\mu\geqslant 0$, and the test statistic $q_{0}$ (eq. (12) in Ref. [100]) is used. Second, the exclusion of $\mu=1$ is tested on the background-only ($\mu=0$) Asimov data set. Here, the null hypothesis is $\mu\geqslant 1$, the alternative is $\mu=0$, and the test statistic $\tilde{q}_{\mu}$ (eq. (16) in Ref. [100]) is used. The CLs criterion [101, 102] is employed, and the null hypothesis is said to be excluded at the 95% confidence level if $CL_{s}=p_{0}/(1-p_{1})<0.05$, where $p_{0,1}$ are the $p$-values under the null and alternative hypotheses.

5 Results

The potential sensitivity to the models discussed in Section 3 is evaluated here following the procedure described above. As already mentioned, two benchmark values are used for the relative $m_{{t\bar{t}}}$ resolution: 10 and 20%. For the integrated luminosity $\mathcal{L}$, three values are considered: 150 fb*-1*, 450 fb*-1*, and 3 ab*-1*, which correspond to the data collected at Run 2 and the targets at Run 3 and and HL-LHC (for a single experiment). In total, this gives six experimental scenarios. It should be noted, however, that the assumed event selection and experimental uncertainties are likely to be not fully representative beyond LHC Run 2.

As already mentioned, we have computed cross sections for the BSM scenarios using the code SusHi (version $1.6.1$) [54, 55] from which we obtain the signal NNLO $K$–factor, for each point of the grid in our two–dimensional parameter planes. The branching ratios and total widths have been computed with HDECAY (version $6-52$) [43, 103, 104]) and FeynHiggs (version 2.14.3) [105, 106].

5.1 The SM with an extra singlet (pseudo)scalar

Figures 9 and 10 show the results for the $\text{SM}+\Phi$ models, where $\Phi=A$ or $H$ is a singlet CP–odd or –even heavy Higgs boson. They are computed, for each of the six experimental scenarios, as functions of the mass of the heavy Higgs boson and its reduced coupling to top quarks. The left plots assume an experimental mass resolution of 10%, and the right plots assume 20%. The top row assumes an integrated luminosity of 150/fb, corresponding to the end of Run 2, the middle row assumes 450/fb, corresponding to the end of Run 3, and the bottom row assumes 3000/fb, corresponding to the completion of the HL-LHC programme. In each case, the expected significance is shown with a colour map, and contours corresponding to values of 1, 3, and 5 $\sigma$ are plotted in white. The red contour indicates the boundary of the region in which each point is excluded at the 95% CL.

Let us consider the implications of the results in Fig. 9 for a pseudoscalar $A$ with a $t\bar{t}$ coupling of the same magnitude as the SM Higgs boson. In the case with a mass resolution of 10%, the 5-$\sigma$ discovery sensitivity reaches 600 GeV already with $\mathcal{L}=150/\text{fb}$, increasing to 750 (880) GeV with $\mathcal{L}=450$ $(3000)/\text{fb}$. With a mass resolution of 20% there is no 5-$\sigma$ discovery sensitivity with 150/fb, and the sensitivities with 450 (3000)/fb are reduced to 580 (650) GeV with 450 (3000)/fb, with a gap around 450 GeV for 450/fb. The potential exclusion ranges from 650 GeV with 20% mass resolution and 150/fb of integrated luminosity to over 1 TeV with 10% mass resolution and 3000/fb of integrated luminosity. It is encouraging that the search for effects in the $t\bar{t}$ mass spectrum has such high sensitivity in the case of a pseudoscalar boson $A$.

In the case of the heavy scalar $H$ shown in Fig. 10 the sensitivity is somewhat reduced compared to the case of the pseudoscalar $A$, because of the lower production rate due first to a smaller value of the form factor $A_{1/2}^{H}$ compared to $A_{1/2}^{A}$ and then to the suppression by a $\beta^{3}$ factor in eqs. (8), as opposed to $\beta$ for the pseudoscalar. In particular, there is a significant loss of sensitivity at low mass that implies, for example, that there are holes in the 5-$\sigma$ discovery potential at low masses $\sim 400$ GeV for 150 and 450/fb of integrated luminosity, even if a mass resolution of 10% is assumed. There is also a degradation in exclusion and discovery sensitivity at large masses. For example, in the most optimistic case of 10% mass resolution and $\mathcal{L}=3/\text{ab}$ the 5-$\sigma$ discovery sensitivity (95% CL exclusion) corresponds to $M_{H}\sim 800$ (980) GeV. Nevertheless, the reach of the search for effects in the $t\bar{t}$ mass spectrum is impressive also in the scalar case.

5.2 The Type II 2HDM

Fig. 11 presents results for a Type II 2HDM in the $[M_{A},\tan\beta]$ plane, assuming $M_{H^{\pm}}\geq\text{max}(M_{A},M_{H})$ and equal scalar and pseudoscalar masses $M_{H}=M_{A}$, with the same mass resolution and integrated luminosity scenarios as previously. As expected, the experimental sensitivity is restricted to relatively low values of $\tan\beta$, and we consider the illustrative case of $\tan\beta=2$. When the mass resolution is 10% and $\mathcal{L}=150/\text{fb}$ points with masses up to 670 GeV can be excluded, but there is no 5-$\sigma$ discovery sensitivity. With 450 (3000)/fb of integrated luminosity, the exclusion region extends to 750 (900) GeV, and there is 5-$\sigma$ discovery sensitivity up to 540 (730) GeV. On the other hand, if the mass resolution is 20%, the expected significance on the line $\tan\beta=2$ does not reach the discovery level in any of the integrated luminosity scenarios, and no point on the line is excluded for 150/fb. However, with $\mathcal{L}=450$ (3000)/fb masses up to 580 (650) GeV can be excluded. This example highlights, therefore, the importance of optimizing the $t\bar{t}$ mass resolution.

In Figures 12, 13, 14 and 15, we present results for a Type II 2HDM in the $[M_{A},\tan\beta]$ plane, assuming $M_{H^{\pm}}\geq\text{max}(M_{A},M_{H})$ and a fixed mass splitting $M_{H}-M_{A}=10$, $50$, $100$ and $200$ GeV, with the same mass resolution and integrated luminosity scenarios as previously.

An increase in the mass separation between $M_{H}$ and $M_{A}$ leads initially to a degradation in the sensitivity, because the $m_{t\bar{t}}$ spectrum is dominated by the structure for the CP–odd state, while the peak–dip structure for the CP–even state lies in the dip for the CP–odd one and partially cancels it out. This is illustrated by the two plots in Fig. 16, one with a perfect resolution, the other with $10\%$ smearing. On the other hand, when the mass separation becomes large enough, the structures from the two states do not overlap so much, and the sensitivity increases again.

5.3 The hMSSM

Fig. 17 presents results for the hMSSM in the $[M_{A},\tan\beta]$ plane, adopting again the same mass resolution and integrated luminosity scenarios as previously. The results for the 95% CL exclusion and 5-$\sigma$ expected significance in these different scenarios are similar to those in the Type II 2HDM analyzed previously.

5.4 Additional vector–like quark contributions to $gg\rightarrow\Phi$

The number of free parameters in the model with additional vector–like quarks (VLQs) is too large for a comprehensive scan. However, it is interesting to see in an illustrative example how the contribution from VLQs in the loop changes the sensitivity compared to the $\text{SM}+\Phi$ model, where only top quarks are considered in the loop. In this scenario, we do not compute the exact NNLO $K$–factor taking into account the contribution of the additional vector–like quarks. Instead, as a first approximation, we set the signal NNLO $K$–factor to two, the typical NNLO correction to the $gg\rightarrow\Phi$ production cross section. Fig. 18 shows results for a single representative experimental scenario with a mass resolution of 10% and 150/fb of integrated luminosity. We consider the case of a CP–even heavy Higgs boson $H$ with $\hat{g}_{\Phi Q\bar{Q}}=\hat{g}_{\Phi t\bar{t}}=1$ and a single VLQ species. We find an increase in significance in the VLQ model compared with the SM+$H$ model over all the $[M_{H},M_{VLQ}]$ plane. Also shown is a dashed line where $M_{H}=2M_{VLQ}$, below which $H\rightarrow Q\bar{Q}$ decays are kinematically allowed. The region where the decays to VLQ are allowed is problematic since the total width becomes then larger than $\Gamma(\Phi\to t\bar{t})$ that we assume to be the total width of the new resonance. However in that region, direct VLQ pair production is likely to provide more distinctive signatures for larger $M_{\Phi}$.

5.5 Stop squark contributions to $gg\rightarrow\Phi$

Within the MSSM, the predictions for $gg\to H$ are sensitive to the parameters of the stop sector, and specifically to the mass of the lighter stop, $\tilde{t}_{1}$. Therefore, we have also studied interference in the case of a light stop in order to highlight the possible effects of stop squark contributions to $gg\rightarrow H$. We obtain the necessary inputs from the Higgs sector and the stop sector using the code FeynHiggs (version 2.14.3) [105, 106], and use them in the “light-stop” MSSM benchmark scenario described in [74, 75, 76]. In this scenario the lighter stop has a mass around $324$ GeV and the heavier stop a mass around $671$ GeV in our scan of the $[M_{A},\tan\beta$] plane. This value of $m_{\tilde{t}_{1}}$ is close to the lower limit from direct LHC searches in the case of a compressed spectrum with a small difference between the masses of the stop and the lightest supersymmetric particle. We present results only for $M_{H}<2m_{\tilde{t}_{1}}$, since direct stop pair production is likely to provide more distinctive signatures for larger $M_{H}$. The NNLO $K$–factor has been computed individually for each point of the grid in the $[M_{A},\tan\beta]$ parameter plane, using the code SusHi (version $1.6.1$) [54, 55].

As seen in Fig. 19, we find significant exclusion and discovery potentials for $M_{H}\lesssim 500$ GeV and smaller values of $\tan\beta$. As expected, the experimental sensitivity is restricted to relatively low values of $\tan\beta$, and we consider the illustrative case of $\tan\beta=2$. In the case when the mass resolution is 10% and the integrated luminosity is 150/fb, the 95% CL exclusion extends to a mass of 550 GeV, with a 5-$\sigma$ discovery sensitivity up to a mass of $\sim 480$ GeV. With $\mathcal{L}=450$ (3000)/fb, the expected exclusion reaches 600 GeV ($\geq 2m_{\tilde{t}_{1}}$), and there is 5-$\sigma$ discovery sensitivity up to 520 (550) GeV. On the other hand, if the mass resolution is 20% there is no discovery sensitivity in any of the integrated luminosity scenarios, and no point is excluded for $\mathcal{L}=150/\text{fb}$. However, values of $M_{H}$ up to 530 (580) GeV can be excluded with 450 (3000)/fb. This example highlights one more time, therefore, the importance of optimizing the $t\bar{t}$ mass resolution.

6 Summary

We have explored in this paper the prospective sensitivity of LHC measurements of the $t{\bar{t}}$ invariant-mass spectrum to various BSM scenarios with additional heavy (pseudo)scalar bosons, via the peak-and-dip features induced by interference between BSM and SM contributions to the production amplitude that are illustrated at the parton level in Fig. 3. The scenarios studied included models with one additional singlet (pseudo)scalar boson, a Type-II 2HDM, the hMSSM, a model with a massive vector–like quark, and models with a light stop squark. Our analysis is based on a realistic assessment of detector performance, taking into account the event selection efficiency shown in Fig. 7 and the $t{\bar{t}}$ invariant-mass resolution, which smears the parton-level features as illustrated in Fig. 8.

We have presented results for benchmark scenarios with resolutions of 10 or 20% and integrated LHC luminosities ranging from 150/fb (corresponding to the Run 2 data set for a single experiment) to 3000/fb (corresponding to the luminosity target for HL-LHC). As could be expected, we find that the ranges of model parameter spaces susceptible to possible exclusion or 5-$\sigma$ discovery at the LHC are larger for smaller $t{\bar{t}}$ invariant-mass resolution and the target HL-LHC luminosity. However, in all the models we find regions of discovery/exclusion at larger values of the (pseudo)scalar-$t{\bar{t}}$ coupling, corresponding in 2HDM models such as variants of the MSSM to smaller values of $\tan\beta$, as illustrated in Fig. 9 and subsequent figures.

Our results underline the importance of taking interference effects into account when interpreting the results of LHC searches for resonance effects in the $t{\bar{t}}$ invariant-mass spectrum. The fact that interferences yield generically dips as well as bumps may serve as a valuable diagnostic tool for distinguishing between interpretations of any features in terms of spin-0 or spin-1 resonances. Although our analysis has not been at a detector–level simulation, we think that we have incorporated sufficient features for our results to provide a realistic estimate of the physics potential of searches for peak-and-dip structures, and we look forward to the results of experimental analyses using the full LHC Run 2 dataset and the higher integrated luminosities to be obtained in the future.

Acknowledgements

This work is supported in part by the Estonian Research Council via Mobilitas Pluss grants. The work of JE was supported in part by the STFC Grant ST/P000258/1. The authors are grateful to Sébastien Brochet and Stéphane Perriès for sharing the MadGraph model employed to simulate the signal process in the $\text{SM}+\Phi$ model, to Sabine Crépé-Renaudin for useful discussions, to Stefan Liebler for valuable help and recommendations regarding the use of SusHi and the “light-stop” MSSM benchmark scenario and to Sébastien Wertz for a valuable suggestion on simulating the interference.