Signatures of vector-like top partners decaying into new neutral scalar or pseudoscalar bosons

TL;DR

This paper investigates the potential for discovering vector-like top partners decaying into a top quark and a new neutral scalar or pseudoscalar at the LHC, proposing optimized search strategies and analyzing their sensitivity.

Contribution

It introduces a simplified model for VLQ decays into new scalars/pseudoscalars, and develops dedicated LHC search strategies for these exotic decay channels.

Findings

Signal regions optimized for $t' ightarrow S t$ decays

Monte Carlo simulations quantify LHC sensitivity to $t'$ and $S$ masses

Potential to surpass current VLQ search strategies at the LHC

Abstract

We explore the phenomenology of models containing one Vector-Like Quark (VLQ), $t'$, which can decay into the Standard Model (SM) top quark, $t$, and a new spin-0 neutral boson, $S$, the latter being either a scalar or pseudoscalar state. We parametrise the underlying interactions in terms of a simplified model which enables us to capture possible Beyond the SM (BSM) scenarios. We discuss in particular three such scenarios: one where the SM state is supplemented by an additional scalar, one which builds upon a 2-Higgs Doublet Model (2HDM) framework and another which realises a Composite Higgs Model (CHM) through partial compositeness. Such exotic decays of the $t'$ can be competitive with decays into SM particles, leading to new possible discovery channels at the Large Hadron Collider (LHC). Assuming $t'$ pair production via strong interactions, we design signal regions optimised for one $t'\rightarrow S t$ transition (while being inclusive on the other \bar{t'} decay, and vice versa), followed by the decay of $S$ into the two very clean experimental signatures $S\rightarrow \gamma \gamma$ and $S\rightarrow Z(\rightarrow \ell^+\ell^-)\gamma$. We perform a dedicated signal-to-background analysis in both channels, by using Monte Carlo (MC) event simulations modelling the dynamics from the proton-proton to the detector level. Under the assumption of BR$(t' \rightarrow S t) = 100\%$, we are therefore able to realistically quantify the sensitivity of the LHC to both the $t'$ and $S$ masses, assuming both current and foreseen luminosities. This approach paves the way for the LHC experiments to surpass current VLQ search strategies based solely on $t'$ decays into SM bosons ($W^\pm, Z$, $h$).

Figures

Tables

Table 1. Table 1 : Fiducial cross section for each mass slice of the two major background processes. For the γ ​ γ + jets 𝛾 𝛾 jets \gamma\,\gamma+\text{jets} background the slices refer to M γ ​ γ bkg subscript superscript 𝑀 bkg 𝛾 𝛾 M^{\mathrm{bkg}}_{\gamma\gamma} while for the Z ​ γ + jets 𝑍 𝛾 jets Z\,\gamma+\text{jets} background the slices refer to M Z ​ γ bkg subscript superscript 𝑀 bkg 𝑍 𝛾 M^{\mathrm{bkg}}_{Z\gamma} at the generator level. The sums of the fiducial cross sections over all slices for each process are also listed together with their estimated value.

Background process		${\sigma }_{\text{fid.}}\left(\gamma \gamma +\text{jets}\right)$ [pb]		${\sigma }_{\text{fid.}}\left(Z\gamma +\text{jets}\right)$ [pb]
50$-$	150 GeV	$69.0$$\pm$	$0.2$	$3.223$$\pm$	$0.003$
150$-$	250 GeV	$3.577$$\pm$	$0.006$	$1.010$$\pm$	$0.001$
250$-$	500 GeV	$(91.3$$\pm$	$0.2)\times 10{}^{-2}$	$(22.56$$\pm$	$0.02)\times 10{}^{-2}$
500$-$	1000 GeV	$(99.2$$\pm$	$0.2)\times 10{}^{-3}$	$(25.43$$\pm$	$0.03)\times 10{}^{-3}$
1000$-$	1500 GeV	$(63.6$$\pm$	$0.2)\times 10{}^{-4}$	$(1.764$$\pm$	$0.002)\times 10{}^{-3}$
Sum		$73.6$$\pm$	$0.1$	$4.486$$\pm$	$0.003$
Estimated total		$74.0$$\pm$	$0.6$	$4.45$$\pm$	$0.03$

Table 2. Table 2 : Selection cuts applied to the S → γ ​ γ → 𝑆 𝛾 𝛾 S\to\gamma\,\gamma signal region. The cuts are described in detail in the text. Refer to section  4.1 for the definition of the objects.

Cut no.	Description
1	$N_{\gamma }\ge 2$
2	$N_{\text{b-jets}}\ge 1$
3
4	($m_S\le 200\text{GeV}$)

Table 3. Table 3 : Signal and background efficiencies in percent following the cuts listed in table  2 , for the γ ​ γ 𝛾 𝛾 \gamma\,\gamma SR and m t ′ = 1000 ​ GeV subscript 𝑚 superscript 𝑡 ′ 1000 GeV m_{t^{\prime}}=1000\,\text{Ge\kern-0.58122ptV} .

$m_S$	$100\text{GeV}$	$200\text{GeV}$	$400\text{GeV}$	$600\text{GeV}$	$800\text{GeV}$
Cut no.	Signal $t\overline{t}\left(S\to \gamma \gamma \right)\left(S\to \gamma \gamma \right)$ efficiency (%)
1	98.1	98.8	99.1	99.0	98.8
2	48.8	47.9	51.0	54.8	60.4
3	35.9	35.9	39.4	42.9	46.4
4	35.8	34.0	39.4	42.9	46.4
Cut no.	Background efficiency (%)
1	15.4	15.4	15.4	15.4	15.4
2	$2.8\times {10}^{-1}$	$2.8\times {10}^{-1}$	$2.8\times {10}^{-1}$	$2.8\times {10}^{-1}$	$2.8\times {10}^{-1}$
3	$5.7\times {10}^{-2}$	$1.2\times {10}^{-2}$	$1.1\times {10}^{-3}$	$2.5\times {10}^{-4}$	$7.1\times {10}^{-5}$
4	$2.0\times {10}^{-2}$	$1.9\times {10}^{-3}$	$1.1\times {10}^{-3}$	$2.5\times {10}^{-4}$	$7.1\times {10}^{-5}$

Table 4. Table 4 : Selection cuts applied to the Z ​ γ 𝑍 𝛾 Z\,\gamma signal region. Details on the cuts are given in the text. Refer to section  4.1 for the definition of the objects.

Cut no.	Description
1	$N_Z\ge 1$
2	$N_{\gamma }\ge 1$
3	$N_{\text{b-jets}}\ge 1$
4	$H_{\text{T}}+E_{\text{T}}^{\text{miss}}>0.3m_{t^{\prime }}$
5

Table 5. Table 5 : Signal and background efficiencies in percent following the cuts listed in table  4 , for the Z ​ γ 𝑍 𝛾 Z\,\gamma SR and m t ′ = 1000 ​ GeV subscript 𝑚 superscript 𝑡 ′ 1000 GeV m_{t^{\prime}}=1000\,\text{Ge\kern-0.58122ptV} .

$m_S$	$130\text{GeV}$	$160\text{GeV}$	$400\text{GeV}$	$800\text{GeV}$
Cut no.	Signal $\left(S\to Z\gamma \right)\left(S\to Z\gamma \right)$ efficiency (%)
1	4.11	4.81	5.80	5.68
2	2.74	3.97	5.39	5.13
3	1.51	2.31	3.27	3.61
4	1.51	2.31	3.27	3.61
5	1.19	1.77	2.43	2.36
Cut no.	Background efficiency (%)
1	21.7	21.7	21.7	21.7
2	4.13	4.14	4.14	4.14
3	0.0731	0.0731	0.0731	0.0731
4	0.0461	0.0463	0.0463	0.0463
5	5.22$\times {10}^{-3}$	8.09$\times {10}^{-3}$	9.39$\times {10}^{-4}$	6.81$\times {10}^{-5}$

Table 6. Table 6 : The background cross section times efficiency σ B γ ​ γ ​ ϵ B γ ​ γ ​ ( m S ) subscript 𝜎 subscript 𝐵 𝛾 𝛾 subscript italic-ϵ subscript 𝐵 𝛾 𝛾 subscript 𝑚 𝑆 \sigma_{B_{\gamma\,\gamma}}\epsilon_{B_{\gamma\,\gamma}}(m_{S}) (in pb) relevant for the γ ​ γ 𝛾 𝛾 \gamma\,\gamma signal region. For this signal region the efficiency is independent of m t ′ subscript 𝑚 superscript 𝑡 ′ m_{t^{\prime}} .

$m_S$ [GeV]	${\sigma }_{B_{\gamma \gamma }}{ϵ}_{B_{\mathrm{SR}}}\left(m_S\right)$ [pb]
$100$	$0.0146$
$200$	$0.00144$
$400$	$8.41\times {10}^{-4}$
$600$	$1.82\times {10}^{-4}$
$800$	$5.23\times {10}^{-5}$
$1000$	$2.14\times {10}^{-5}$
$1200$	$7.64\times {10}^{-6}$
$1400$	$3.10\times {10}^{-6}$

Table 7. Table 7 : The background cross section times efficiency σ B Z ​ γ ​ ϵ B Z ​ γ ​ ( m S , m t ′ ) subscript 𝜎 subscript 𝐵 𝑍 𝛾 subscript italic-ϵ subscript 𝐵 𝑍 𝛾 subscript 𝑚 𝑆 subscript 𝑚 superscript 𝑡 ′ \sigma_{B_{Z\,\gamma}}\epsilon_{B_{Z\,\gamma}}(m_{S},m_{t^{\prime}}) (in pb) relevant for the Z ​ γ 𝑍 𝛾 Z\,\gamma signal region.

$m_S$ [GeV]	$m_{t^{\prime }}$ [GeV]
	600	800	1000	1200	1400	1600
130	5.87$\times {10}^{-4}$	3.94$\times {10}^{-4}$	2.39$\times {10}^{-4}$	1.39$\times {10}^{-4}$	7.32$\times {10}^{-5}$	5.15$\times {10}^{-5}$
160	7.61$\times {10}^{-4}$	5.90$\times {10}^{-4}$	3.70$\times {10}^{-4}$	2.34$\times {10}^{-4}$	1.54$\times {10}^{-4}$	9.65$\times {10}^{-5}$
200	4.79$\times {10}^{-4}$	4.37$\times {10}^{-4}$	3.47$\times {10}^{-4}$	2.47$\times {10}^{-4}$	1.47$\times {10}^{-4}$	9.48$\times {10}^{-5}$
400	4.55$\times {10}^{-5}$	4.42$\times {10}^{-5}$	4.30$\times {10}^{-5}$	4.10$\times {10}^{-5}$	3.75$\times {10}^{-5}$	3.24$\times {10}^{-5}$
600		9.98$\times {10}^{-6}$	9.88$\times {10}^{-6}$	9.72$\times {10}^{-6}$	9.39$\times {10}^{-6}$	8.96$\times {10}^{-6}$
800			3.12$\times {10}^{-6}$	3.12$\times {10}^{-6}$	3.05$\times {10}^{-6}$	3.02$\times {10}^{-6}$
1000				1.16$\times {10}^{-6}$	1.11$\times {10}^{-6}$	1.11$\times {10}^{-6}$
1200					5.01$\times {10}^{-7}$	4.94$\times {10}^{-7}$
1400						2.17$\times {10}^{-7}$

Equations

$${\mathcal{L}}_{\text{BSM}}$$

$${\Gamma }_{t^{\prime }\to St}$$

$$\times {\left(1+x_t^4+x_S^4-2x_t^2-2x_S^2-2x_t^2{x}_S^2\right)}^{\frac{1}{2}},$$

$${\mathcal{L}}_{\mathrm{kin}}\supset \bar{T}\left(i/D-M\right)T+\frac{1}{2}\left({\partial }_{\mu }S\right)\left({\partial }^{\mu }S\right)-\frac{1}{2}{m}_S^2{S}^2,$$

$${\mathcal{L}}_{\mathrm{int}}$$

$${\mathcal{L}}_t\supset \left(\begin{array}{cc}{\bar{\tilde{t}}}_L& {\bar{T}}_L\end{array}\right)\left(\begin{array}{cc}{m}_{\tilde{t}}& m_1\\ m_2& M\end{array}\right)\left(\begin{array}{c}{\tilde{t}}_R\\ T_R\end{array}\right)+\text{h.c.},$$

$$\left(\begin{array}{c}{t}_{L,R}\\ t_{L,R}^{\prime }\end{array}\right)=\left(\begin{array}{cc}{c}_{L,R}& -s_{L,R}\\ s_{L,R}& c_{L,R}\end{array}\right)\left(\begin{array}{c}{\tilde{t}}_{L,R}\\ T_{L,R}\end{array}\right),$$

$$\tan (2{\theta }_L)=\frac{2\left(m_{\tilde{t}}{m}_2+M{m}_1\right)}{M^2-m_{\tilde{t}}^2-m_1^2+m_2^2},\tan (2{\theta }_R)=\frac{2\left(m_{\tilde{t}}{m}_1+M{m}_2\right)}{M^2-m_{\tilde{t}}^2+m_1^2-m_2^2}.$$

$${\kappa }_L^S={\left({\lambda }_S^a{s}_L{c}_R+{\lambda }_S^b{s}_L{s}_R\right)}^{\ast },{\kappa }_R^S={\lambda }_S^a{c}_L{s}_R-{\lambda }_S^b{c}_L{c}_R,$$

$${\kappa }_t=\mathrm{Re}\left(-{\lambda }_S^a{s}_L{s}_R+{\lambda }_S^b{s}_L{c}_R\right),{\tilde{\kappa }}_t=\mathrm{Im}\left(-{\lambda }_S^a{s}_L{s}_R+{\lambda }_S^b{s}_L{c}_R\right).$$

$$\mathrm{BR}(S\to \gamma \gamma )\simeq \frac{{\Gamma }_{S\to \gamma \gamma }}{{\Gamma }_{S\to gg}}=\frac{8{\alpha }_{EM}^2}{9{\alpha }_S^2}\approx \mathrm{0.004.}$$

$$\tan {\theta }_L=\frac{m_{t^{\prime }}}{m_t}\tan {\theta }_R,\frac{{\xi }_T}{y_t}=s_L{c}_L\frac{m_{t^{\prime }}^2-m_t^2}{m_t{m}_{t^{\prime }}},$$

$$m_{t^{\prime }}^2$$

$$\mathcal{H}\oplus S\equiv \left(\begin{array}{cc}{H}^{0\ast }& H^{+}\\ -H^{+\ast }& H^0\end{array}\right)\oplus S\in (2,2)\oplus (1,1)=5.$$

$$m_t=\frac{y_L{y}_{R}fv}{\sqrt{2}\hat{M}}+\mathcal{O}\left(v^2/f^2\right),$$

$$m_{t^{\prime \prime }}=M+\mathcal{O}\left(v^2/f^2\right),$$

$$\mathcal{L}=-i{y}_{R}S{\bar{t}}_L^{\prime }{t}_R-\frac{i{y}_{L}vM}{\sqrt{2}f\hat{M}}S{\bar{t}}_R^{\prime }{t}_L+\text{mbox}h.c.,$$

$${\kappa }_R^S=-i{y}_R,{\kappa }_L^S=-\frac{i{y}_{L}vM}{\sqrt{2}f\hat{M}}.$$

$${\mathcal{L}}_{SVV}=\frac{A\cos\theta}{16\pi^{2}f}\,S\,\bigg{(}\frac{g^{2}-g^{\prime 2}}{2}Z_{\mu\nu}\widetilde{Z}^{\mu\nu}+gg^{\prime}F_{\mu\nu}\widetilde{Z}^{\mu\nu}+g^{2}W^{+}_{\mu\nu}\widetilde{W}^{-\mu\nu}\bigg{)},$$

$${\tilde{\kappa }}_W=-{\tilde{\kappa }}_B=\frac{Av}{2f}\cos \theta .$$

$$S\to \{\gamma \gamma ,Z\gamma ,WW,ZZ\}.$$

$$B_{\mathrm{SR}}(m_S,m_{t^{\prime }})=L{\sigma }_{B_{\mathrm{SR}}}{ϵ}_{B_{\mathrm{SR}}}(m_S,m_{t^{\prime }})$$

$$S_{\mathrm{SR}}=L\sigma (m_{t^{\prime }})$$

$$z=\sqrt{2}{\left\{(S+B)\ln \left[\frac{(S+B)(B+{\sigma }_b^2)}{B^2+(S+B){\sigma }_b^2}\right]-\frac{B^2}{{\sigma }_b^2}\ln \left[1+\frac{{\sigma }_b^{2}S}{B(B+{\sigma }_b^2)}\right]\right\}}^{1/2},$$

$$V$$

$$+{\lambda }_3\left({\Phi }_1^{†}{\Phi }_1\right)\left({\Phi }_2^{†}{\Phi }_2\right)+{\lambda }_4\left({\Phi }_1^{†}{\Phi }_2\right)\left({\Phi }_2^{†}{\Phi }_1\right)+\frac{{\lambda }_5}{2}\left[{\left({\Phi }_1^{†}{\Phi }_2\right)}^2+{\left({\Phi }_2^{†}{\Phi }_1\right)}^2\right].$$

$$\displaystyle H_{1}=\frac{1}{\sqrt{2}}\left(\begin{array}[]{c}\sqrt{2}\,G^{+}\\ v+\varphi^{0}_{1}+iG^{0}\\ \end{array}\right),\quad H_{2}=\frac{1}{\sqrt{2}}\left(\begin{array}[]{c}\sqrt{2}\,H^{+}\\ \varphi^{0}_{2}+iA\\ \end{array}\right),$$

$$\displaystyle\left(\begin{array}[]{c}h\\ H\\ \end{array}\right)=\left(\begin{array}[]{cc}s_{\beta-\alpha}&c_{\beta-\alpha}\\ c_{\beta-\alpha}&-s_{\beta-\alpha}\\ \end{array}\right)\left(\begin{array}[]{c}\varphi^{0}_{1}\\ \varphi^{0}_{2}\\ \end{array}\right),$$

$$-{\mathcal{L}}_Y\supset y_T{\overline{Q}}_L{\tilde{H}}_2{T}_R+{\xi }_T{\overline{Q}}_L{\tilde{H}}_1{T}_R+M{\overline{T}}_L{T}_R,$$

$$\displaystyle{\mathcal{M}}=\left(\begin{array}[]{cc}\frac{y_{t}v}{\sqrt{2}}&\frac{\xi_{T}v}{\sqrt{2}}\\ 0&M\\ \end{array}\right),$$

$$\displaystyle\left(\begin{array}[]{c}t_{L,R}\\ t^{\prime}_{L,R}\\ \end{array}\right)=\left(\begin{array}[]{cc}c_{L,R}&-s_{L,R}\\ s_{L,R}&c_{L,R}\\ \end{array}\right)\left(\begin{array}[]{c}\widetilde{t}_{L,R}\\ T_{L,R}\\ \end{array}\right)=U_{L,R}\left(\begin{array}[]{c}\widetilde{t}_{L,R}\\ T_{L,R}\\ \end{array}\right)$$

$$\tan (2{\theta }_L)=\frac{\sqrt{2}Mv{\xi }_T}{M^2-\frac{y_t^2{v}^2}{2}-\frac{{\xi }_T^2{v}^2}{2}},\tan (2{\theta }_R)=\frac{y_t{\xi }_T{v}^2}{M^2-\frac{y_t^2{v}^2}{2}+\frac{{\xi }_T^2{v}^2}{2}},$$

Full text

1 Introduction

During Run II at the LHC, the ATLAS and CMS experiments have collected almost ${150\,\mathrm{\mathrm{fb}^{-1}}}$ and ${180\,\mathrm{\mathrm{fb}^{-1}}}$ of data, respectively, at a centre-of-mass (CM) energy of ${13\,\mathrm{\text{Te\kern-0.43057ptV}}}$. These data are now being analysed by the collaborations and, so far, no significant deviations from the SM have been recorded. This has significantly restricted the parameter space of the most common scenarios attempting to solve the hierarchy problem of the SM, such as supersymmetry and compositeness. Yet, it is important to find a viable solution to this flaw of the SM. This is inevitably connected to studying both top quark and Higgs boson dynamics, as the hierarchy problem of the SM originates from their mutual interactions. A pragmatic approach is to investigate BSM scenarios in which either of or both the top and Higgs sectors of the SM are enlarged through the presence of companions to the SM states ($t$ and $h$), by which we mean additional spin-1/2 and spin-0 states, respectively, with the same electromagnetic (EM) charge but different mass (naturally heavier) and possibly different quantum numbers as well.

Some guidance in exploring the various BSM possibilities in this respect is afforded by experimental measurements of observables where both the top quark and the SM-like Higgs boson enter. On the one hand, a sequential fourth family of chiral SM quarks is strongly constrained indirectly from Higgs boson measurements due to their non-decoupling properties Eberhardt:2012gv , while VLQs (which transform as triplets under colour but whose left- and right-handed components have identical electroweak (EW) quantum numbers) can evade these bounds easily. On the other hand, the possibility of the existence of additional Higgs bosons has not been excluded by experimental data and may well be theoretically motivated by the fact that neither the matter nor the gauge sectors are minimal. Moreover, the Higgs sector is extended in any supersymmetric model or in the 2HDM.

Similarly, any model in which a Higgs boson arises as a pseudo-Nambu-Goldstone Boson (pNGB), other than the minimal model based on the symmetry breaking pattern $\text{SO}(5)/\text{SO}(4)$, will include additional light (pseudo)scalars that might well have eluded direct searches due to their reduced couplings to the EW bosons and top quark.

Hence, it is of some relevance to assess the viability at the LHC of BSM models with both top quark partners (of VLQ nature) and companion scalar or pseudoscalar particles (both charged and neutral). In fact, it is particularly intriguing to investigate the possibility of isolating experimental signatures where the two particle species interact with each other, namely, when the $t^{\prime}$ decays into a new (pseudo)scalar.

So far, collider searches for a VLQ companion to the SM top quark Buchkremer:2013bha ; Aguilar-Saavedra:2013qpa have mostly been carried out under the assumption that it decays exclusively into SM particles, namely, a heavy quark ($b,t$) and a boson ($W^{\pm},Z,h$), compatibly with the EM charge assignments. Specifically, for the case of a top-like VLQ, $t^{\prime}$, the decays considered are $t^{\prime}\to Z\,t$, $t^{\prime}\to h\,t$ and $t^{\prime}\to W^{+}\,b$, with varying branching ratios ($\mathrm{BRs}$) adding up to 100%, see e.g. Aaboud:2018pii ; Aaboud:2018wxv ; Aaboud:2018xpj ; Aaboud:2018uek ; Aaboud:2018saj ; Aaboud:2018xuw ; Sirunyan:2018qau ; Sirunyan:2018yun ; Sirunyan:2017pks ; CERN-EP-2019-129 .

It is thus important to ask how the presence of exotic decay channels of VLQs can affect the current bounds and whether these might actually be promising discovery channels on their own. This question has been asked in similar contexts in various preceding works Serra:2015xfa ; Anandakrishnan:2015yfa ; Banerjee:2016wls ; Dobrescu:2016pda ; Aguilar-Saavedra:2017giu ; Chala:2017xgc ; Colucci:2018vxz ; Banerjee:2018fsx ; Han:2018hcu ; Kim:2019oyh , each concentrating on a specific BSM construction. Here, in contrast, we follow the approach of Bizot:2018tds , which adopts a set of simplified scenarios based on effective Lagrangians (motivated by compositeness).

In our paper, we build upon this last work, by adopting a simplified scenario which contains, above and beyond the SM particle spectrum, a top-like VLQ, $t^{\prime}$, as well as an additional scalar (or pseudoscalar) particle, $S$, in turn leading to the new decay channel $t^{\prime}\to S\,t$. As for the decay modes of $S$, we will concentrate on two of the experimentally cleanest channels accessible at the LHC, namely, $S\to\gamma\,\gamma$ and $S\to Z\,\gamma$, with the $Z$ boson decaying in turn into electrons or muons. We will show in section 2 that there exist well motivated phenomenological scenarios where these can indeed be decay modes with significant $\mathrm{BRs}$, for the case of both fundamental and composite Higgs states. In section 3 we estimate LHC constraints using published ATLAS and CMS searches in $\gamma\,\gamma$ and $Z\,\gamma$ final states while in section 4 we will describe our MC simulations, based on the pair production process $p\,p\to t^{\prime}\,\bar{t}^{\prime}$, followed by the decay chains $t^{\prime}\to S(\to\gamma\,\gamma)\,t$ or $t^{\prime}\to S(\to Z\,\gamma)\,t$, with the $\bar{t}^{\prime}$ treated inclusively (and vice versa). Section 5 is then dedicated to interpreting the ensuing MC results in three theoretical scenarios embedding a $t^{\prime}$ alongside additional (pseudo)scalar states focusing on cases with $\mathrm{BR}({t^{\prime}\to S\,t})=100\%$, while in section 6 we conclude.

2 The simplified model

The purpose of this section is to present the relevant details about the class of models whose phenomenology we aim to study. We begin with a general description of a simplified model that captures all relevant features. This is the model used for the analysis in section 4. We then justify the use of this simplified model by introducing three more specific models that can all be described with the same generic Lagrangian by a mapping of the fields and the couplings, provided that the processes considered in this paper are studied.

As discussed in the introduction, we are interested in exotic decays of a top partner $t^{\prime}$ (of mass $m_{t^{\prime}}$) into the ordinary top quark $t$ and a scalar (or pseudoscalar) generically denoted by $S$ (of mass $m_{S}$) in the simplified model. We can thus augment the SM Lagrangian $\mathcal{L}_{\rm SM}$ by the following interaction Lagrangian with operators up to dimension five involving these two additional fields,

[TABLE]

Here $\kappa^{S}_{L}$ and $\kappa^{S}_{R}$ are the Yukawa couplings of the $S$ to the $t$ and $t^{\prime}$. In the second line, $f$ sums over all SM fermions (including the top $t$) and $\kappa_{f}$ is the dimensionless reduced Yukawa coupling. In the last line $V_{\mu\nu}$ denotes the field strengths of the $\text{U}({1})$Y, $\text{SU}(2)$L and $\text{SU}(3)$C gauge bosons $B_{\mu},W_{\mu},G_{\mu}$ in the gauge eigenbasis, $g_{V}$ is the associated gauge coupling ($g^{\prime},g,g_{s}$ respectively) and $\widetilde{V}_{\mu\nu}=(1/2)\epsilon_{\mu\nu\rho\gamma}V^{\rho\gamma}$ is the dual field strength tensor. The coefficients $\tilde{\kappa}_{V}$ and $\kappa_{V}$ are couplings associated with dimension-five operators and are typically generated by loops of heavy particles or via anomalies. The couplings $\lambda_{V}$ for any gauge boson $V$ are only generated if $S$ is charged under some of the SM gauge groups and gets a vacuum expectation value (VEV) or if it mixes with such states, e.g., the Higgs boson. Since $\text{SU}(3)_{\mathrm{C}}$ and $\text{U}({1})_{\mathrm{EM}}$ are unbroken for the strong and EM interactions, $\lambda_{V}=0$ for the respective gauge bosons. We choose to normalise all terms with only one dimensionful parameter, the VEV $v=246\,\text{Ge\kern-0.6458ptV}$.

In practice, we consider an $S$ state of either scalar or pseudoscalar nature, but not a mixture. We therefore do not consider CP-violation in this paper. This means that either $\tilde{\kappa}_{V}$ or $\tilde{\kappa}_{f}$ are zero, in the scalar case, or $\kappa_{V}$, $\lambda_{V}$ and $\kappa_{f}$ are zero, in the pseudoscalar case.

The total widths of $t^{\prime}$ and $S$ are kept as free parameters in the simulation as an indication that other interactions and other states might be present. These interactions are not explicitly required to describe the process $p\,p\to t^{\prime}\,\bar{t}^{\prime}\to SSt\bar{t}$ apart from their contribution to the total widths. Here we only report the analytic expression for the partial width of the exotic $t^{\prime}$ decay, specifically.

[TABLE]

where $x_{t}\equiv m_{t}/m_{t^{\prime}}$ and $x_{S}\equiv m_{S}/m_{t^{\prime}}$. This formula is valid for decays into both scalar and pseudoscalar $S$.

This defines the simplified model that will be used in the rest of this paper. Let us now briefly discuss three specific examples of models that motivate the use of the above simplified model and the mapping between the former and the latter. The results in this paper, given in terms of the simplified model above, can then easily be reinterpreted in terms of each model, if needed. In a forthcoming paper, we will specify these models in more detail and will discuss their specific phenomenology.

2.1 Example 1: adding a VLQ and a scalar to the SM

In order to illustrate how a particular model can be related to the phenomenological simplified model (eq. (1)), we will first present a simple model of top-quark partial compositeness (PC) in some detail. The model consists of the SM extended by a top partner VLQ and a scalar singlet. In this model the top quark acquires its mass via the mixing with the top partner. This model is not intended as a complete, realistic model, but provides an example of a model with an additional scalar $S$ that is neutral under the SM gauge group. We will only be concerned with the couplings between the top quarks and $S$, leaving the coupling inducing the decay of the $S$ to SM states as in eq. (1).

We denote the gauge eigenstates in the top sector by $\widetilde{t}_{L}$, $\widetilde{t}_{R}$ and $T$. The notation $\widetilde{t}_{L/R}$ is to prevent confusion with the mass eigenstates that are to be denoted by $t$ and $t^{\prime}$. The Lagrangian for this model before EW symmetry breaking (EWSB) can be written as

[TABLE]

[TABLE]

where the SM Higgs doublet is denoted by $H$ with $\widetilde{H}=i\sigma_{2}H^{*}$. The SM Yukawa coupling for the top quark is here denoted by $\widetilde{y}$ and $Q_{L}$ is the left-handed quark doublet of the third generation. The couplings $\lambda_{S}^{a,b}$ are real if $S$ is a scalar and purely imaginary if $S$ is a pseudoscalar. The mass $m_{2}$ is a non-diagonal entry in the mass matrix of eq. (5). The remaining couplings are dimensionless. After EWSB, we have a mass matrix

[TABLE]

where we defined $m_{\widetilde{t}}=\widetilde{y}v/\sqrt{2}$ and $m_{1}=\lambda_{1}v/\sqrt{2}$. The mass matrix can be diagonalised by bi-orthogonal rotations by the angles $\theta_{L,R}$, separately for left- and right-handed fermions, as follows (where $s_{X}\equiv\sin\theta_{X}$ and $c_{X}\equiv\cos\theta_{X}$)

[TABLE]

where $\{t,t^{\prime}\}$ are the mass eigenstates and the mixing angles are given by

[TABLE]

The mass eigenvalues $m_{t}$ and $m_{t^{\prime}}$ are found by computing the eigenvalues. This model can be mapped to the simplified model Lagrangian in eq. (1) by performing the rotation in eq. (6) inside eq. (4). Focusing on the mixing terms yields

[TABLE]

while for the coupling to the top we have

[TABLE]

There is also a diagonal term involving the $t^{\prime}$, which is proportional to $\lambda^{a}_{S}c_{L}c_{R}+\lambda^{b}_{S}c_{L}s_{R}$. It is not included in the simplified model, but instead generates a contribution to the effective coefficients $\kappa_{V}$ and $\widetilde{\kappa}_{V}$ from loop diagrams.

Let us also briefly discuss the decays of the $t^{\prime}$ and $S$ in this model. The $t^{\prime}$ has both the standard and non-standard decay channels discussed above, where the width of the $t^{\prime}\to S\,t$ channel is given by eq. (2) with the couplings defined in eq. (8). The scalar can, in general, decay into the final states $gg,\,\gamma\gamma,\,Z\gamma,\,ZZ,\,WW$ and $t\bar{t}$. We always assume $m_{S}<m_{t^{\prime}}$, which forbids the decay $S\to t^{\prime}\bar{t}^{\prime}$. Apart from the $t\bar{t}$ channel, all the other decays are generated by loops of the $t$ and $t^{\prime}$.

We may now examine the decay of the $t^{\prime}$ and $S$ depending on the coupling of $T_{L}$ with $T_{R}$ and $\widetilde{t}_{R}$. The $t^{\prime}\to S\,t$ decay is induced by the $\lambda_{S}^{a}$ and $\lambda_{S}^{b}$ couplings. If we are interested in a large $\mathrm{BR}({t^{\prime}\to St})$, we may achieve that easily in a wide region of parameter space by considering suitable values of these couplings. For example, when the $T_{L}$ couples to $\widetilde{t}_{R}$ (i.e., $\lambda_{S}^{a}=0,\lambda_{S}^{b}\neq 0$), a small $\lambda_{S}^{b}$ can induce large $\mathrm{BR}({t^{\prime}\to St})$ as the $\lambda_{S}^{b}$ part of $\kappa^{S}_{R}$ in eq. (8) is proportional to $c_{L}c_{R}\sim 1$. If the $T_{L}$ only couples to $T_{R}$ (i.e., $\lambda_{S}^{a}\neq 0,\lambda_{S}^{b}=0$), a large $\mathrm{BR}({t^{\prime}\to St})$ is realised when $\lambda_{S}^{a}$ is sufficiently large, as the partial width is proportional to $(\lambda_{S}^{a}/\lambda_{t})^{2}$. However, this will also increase the $s$-channel production of $S$ through $gg$ fusion, therefore, this scenario is heavily constrained by the $gg\to S\to\gamma\gamma$ resonance search data from the LHC. In figure 1, we show the $\mathrm{BRs}$ of $t^{\prime}$ for a specific benchmark point where the $t^{\prime}\to S\,t$ channel has a $\mathrm{BR}$ of almost 100%.

As for the $S$ decay, the $S\to gg$ channel dominates if the $t\bar{t}$ decay is not kinematically allowed, $m_{S}<2m_{t}$. The total decay width is governed by $\Gamma_{S\to gg}$, and hence the branching ratio in the $\gamma\gamma$ channel is approximately

[TABLE]

Despite the small BR, the $S\to\gamma\gamma$ decay is a clean and well motivated channel. For instance, in the search for a VLQ decaying into a Higgs boson and a top, $t^{\prime}\to ht$, the $h\to\gamma\gamma$ decay channel (which has a BR of 0.23%) is still sensitive Khachatryan:2015oba . We also note that there is no dedicated di-jet search, $t^{\prime}\to St\to ggt$, although it has been recently proposed in ref. Cacciapaglia:2019zmj . The current bounds estimated by a recast of R-parity violating (RPV) supersymmetry searches Aad:2015lea are not competitive. Other loop induced channels are more suppressed than $S\to\gamma\gamma$. For example, the partial widths of $S\to Z\,\gamma$ and $S\to ZZ$, modulo negligible $m_{Z}$ corrections, are $2\tan^{2}\theta_{W}\Gamma_{S\to\gamma\gamma}$ and $\tan^{4}\theta_{W}\Gamma_{S\to\gamma\gamma}$, respectively.

For $m_{S}>2m_{t}$, the tree-level $S\to$ $t\bar{t}$ channel usually dominates over the loop induced decays. However, in a region of parameter space, the $t\bar{t}$ decay can be tuned down by suitable values of the off-diagonal entries in the mass matrix in eq. (5). We find that, when $\sin\theta_{L}\ll\sin\theta_{R}$ (or equivalently $m_{1}\ll m_{2}$), the effective $St\bar{t}$ coupling, depending on the $\lambda_{S}^{a}$ and $\lambda_{S}^{b}$ couplings, is not sufficiently large to compete with the loop induced decays of $S$. The six tops final state via $t^{\prime}\to S\,t\to tt\bar{t}$ has been discussed in ref. Han:2018hcu with both a recast from current searches and a dedicated analysis.

2.2 Example 2: adding a VLQ to the 2HDM

The 2HDM (see Branco:2011iw for a review) is widely used as a minimal model for an extended Higgs sector that goes beyond additional singlet scalars. With additional vector-like top partners (see Badziak:2015zez ; Angelescu:2015uiz ; Arhrib:2016rlj for previous work), the 2HDM may be seen as the low-energy manifestation of a composite Higgs scenario, such as in DeCurtis:2018zvh . Specifically, we here consider a vector-like top partner $T$ with charge $+2/3$ in the singlet representation of the SM EW group. We further consider Yukawa couplings of the SM quarks of Type-II, i.e., that the up- and down-type quarks couple to different doublets.

The Higgs sector of the 2HDM has an additional neutral scalar $H$, a pseudoscalar $A$ and a charged $H^{\pm}$ state. This enables us to obtain simple formulae where either $H$ or $A$ can play the role of $S$ in the simplified model Lagrangian in eq. (1). The details of the model and the involved parameters as well as the mapping onto the simplified model Lagrangian of eq. (1) are discussed in appendix A.1. Let us here only discuss the mixing of the physical top quark $t$ and top partner $t^{\prime}$.

The physical mass of the heavy top, $m_{t^{\prime}}$, is different from the mass $M$ of the vector-like $T$ due to $t$–$T$ mixing. The mass matrix can be diagonalised in the same way as in eq. (6) to obtain the physical states $(t_{L,R},t^{\prime}_{L,R})$ in terms of the gauge eigenstates $(\widetilde{t}_{L,R},T_{L,R})$. The mixing angles $\theta_{L}$ and $\theta_{R}$ are not independent parameters and we can derive similar relations to eq. (6) (see eq. (50)), in terms of the Yukawa couplings $y_{t}$ and $\xi_{T}$ that couple the left-handed quark doublet $Q_{L}$ to the right-handed SM top $\widetilde{t}_{R}$ and the vector-like $T_{R}$, respectively (see eq. (37) and eq. (68)). The two mixing angles in this case satisfy Arhrib:2016rlj

[TABLE]

while the mass of the $t^{\prime}$ is related to the Lagrangian parameters and the physical top quark mass via

[TABLE]

The $t^{\prime}$–$t$ interaction can thus be described by three independent physical parameters: two quark masses, $m_{t}$ and $m_{t^{\prime}}$, and a mixing angle, $s_{L}=\sin\theta_{L}$.

In the 2HDM with a VLQ, the scalar $S$ is an additional Higgs boson. The dimension-five operators in eq. (1) are then generated through loops and in general $S$ can be produced through $gg\to S$. It can then decay in all the bosonic channels that we consider in this paper and, in addition, in fermionic ones. (The BRs in this model are discussed in section 5.) These channels give rise to constraints from all the usual collider observables. In addition, the scalar sector of this model is subject to the same unitarity, perturbativity and vacuum stability constraints as the usual 2HDM Branco:2011iw ; Kanemura:1993hm . The Yukawa coupling $y_{t}$ is constrained from unitarity to be less than $4\pi$, while $\xi_{T}$ is a derived quantity. Since the new top partner will contribute to gauge boson self energies, the mixing angle $\theta_{L}$ can be constrained from EW Precision Tests (EWPTs) such as the $S$ and $T$ parameters. Based on ref. Arhrib:2016rlj , such bounds require the mixing angle $\theta_{L}$ to be in the range $(-0.15,+0.15)$. However, the constraints coming from BR$(b\to s\gamma)$ are the most relevant ones, as the mixing angle is restricted to be in the range $(-0.1,+0.1)$ for large $m_{t^{\prime}}$, i.e., around ${1\,\mathrm{\text{Te\kern-0.43057ptV}}}$.

2.3 Example 3: realisation in partial compositeness

Lastly, we present a Composite Higgs Model (CHM), which motivates the analysis in this paper by having a top partner with enhanced exotic decay mode and a pseudoscalar with dominant $Z\,\gamma$ decay. The model is closely related to one of the earliest non-minimal models of composite Higgs with fermionic partial compositeness Gripaios:2009pe , based on the coset space $\text{SU}(4)/\text{Sp}(4)$, where Sp is the symplectic group. The usual Higgs field $\mathcal{H}$ is a bi-doublet of $\text{SU}(2)_{L}\times\text{SU}(2)_{R}$, which together with a singlet $S$ (usually denoted by $\eta$ in the CHM literature) forms the five dimensional anti-symmetric irreducible representation of $\text{Sp}(4)$,

[TABLE]

This scenario has the further appeal of belonging to a class of models that can be obtained from an underlying gauge theory with fermionic matter Barnard:2013zea ; Ferretti:2013kya and the additional features arising from this fact have been studied in, e.g., Belyaev:2016ftv . Here, however, we want to focus on the bare bones of the model, namely the above-mentioned coset structure with the addition of one fermionic partner $\Psi$. (We only consider partial compositeness in the top sector).

The fermionic sector also consists of a bi-doublet and a singlet in the $\mathbf{5}$ of $\text{Sp}(4)$. We will see that, as already anticipated in Bizot:2018tds (see also Serra:2015xfa ), the possible decay patterns of the fermionic partners are richer than what is usually considered in current searches and, in particular, the lightest top-partner has an enhanced decay into the exotic channel $t^{\prime}\to S\,t$.

To summarise, in addition to the SM fields the model has an additional pseudoscalar $S$, three top partners $T,T^{\prime},\widetilde{T}$ (all of electric charge $+2/3$), a bottom partner $B$ (charge $-1/3$) and an additional coloured fermion $X$ of charge $+5/3$. Like in the previous example models, all of these fermions are vector-like Dirac spinors, to be thought of as in the gauge eigenbasis, i.e., before their mass matrices are diagonalised. The difference here is that there are more than one new fermion.

The mixing with the third family quarks of the SM depends on how they are embedded in a representation of $\text{SU}(4)$. We choose this embedding such that the custodial symmetry of Agashe:2006at is preserved, see appendix A.2 for details. In addition, the choice of having an elementary $\widetilde{t}_{R}$ distinguishes this model from similar ones studied in DeSimone:2012fs , where the $\widetilde{t}_{R}$ was taken to be fully composite. The elementary $\widetilde{t}_{R}$ seems more appealing, since chiral fermions are notoriously difficult to obtain from underlying strongly coupled theories. We do not address the origin of the bottom quark mass in this work, which would add additional model dependence that is not relevant for the experimental signatures of interest. See appendix A.2 for more details on the construction of the model and the singular value decomposition of the mass matrix.

We end up with four top quark mass eigenstates, which we denote, in increasing mass order, by $t,t^{\prime},t^{\prime\prime}$ and $t^{\prime\prime\prime}$. Here $t$ is the known SM top quark of mass $m_{t}=173\,\text{Ge\kern-0.6458ptV}$. We diagonalise the mass matrix numerically, but a perturbative expansion for the masses gives some insight into the mass spectrum. We find (see appendix A.2)

[TABLE]

where $M$ is the mass parameter of the $\Psi$, $y_{L}$ and $y_{R}$ are the respective couplings of the $Q_{L}$ and $\widetilde{t}_{R}$ to the $\Psi$ and pNGBs while $f$ is the “pion decay constant” of the strongly coupled theory. We also defined $\widehat{M}=\sqrt{M^{2}+y_{L}^{2}f^{2}}$. The mass of the bottom partner (mostly aligned with $B$) turns out to be of the same order as that of the heaviest top partner $m_{t^{\prime\prime\prime}}$, while $X$ has mass equal to $M\equiv m_{t^{\prime}}$ since it does not mix with anything.

Substituting the mass eigenstates (see appendix A.2) into the Lagrangian and considering the coupling that mixes the two lightest eigenstates $t$ and $t^{\prime}$ with the pNGBs, we see that no mixing with the Higgs field $h$ arises, while the $S$ couples, up to terms of order ${\mathcal{O}}\left(v^{2}/f^{2}\right)$, as

[TABLE]

allowing us to match the models with the parameters of the phenomenological Lagrangian eq. (1)

[TABLE]

From the analysis of the spectrum and of the couplings, we see that we can concentrate on a model with two mass degenerate VLQs $t^{\prime}$ and $X$, with $\sim 100\%$ branching ratios $X\to W^{+}\,t$ and $t^{\prime}\to S\,t$. The decay modes of $t^{\prime}$ to SM vector bosons are highly suppressed, $t^{\prime}$ being a singlet of $\text{SU}(2)_{L}\times\text{SU}(2)_{R}$. For this model, it is thus crucial to understand whether the BSM decay $t^{\prime}\to S\,t$ can compete with the SM decay $X\to W^{+}\,t$ whose signatures have been looked for at the LHC Aaboud:2018xpj providing bounds to the model parameter $M>1.2\,\text{Te\kern-0.43057ptV}$. We address this question in this work. Just above the $t^{\prime}$ mass scale there is a further top partner, $t^{\prime\prime}$, with more diverse and model dependent decay modes, so it is likely to be less relevant to experimental searches. The last top partner $t^{\prime\prime\prime}$ and the $B$ are heavy and can be ignored altogether.

The coupling of the $S$ to gauge bosons can be motivated by the analysis of the underlying gauge theory Barnard:2013zea ; Ferretti:2013kya and is given at leading order by the Lagrangian

[TABLE]

where the “Abelian” field strength tensors are defined as $V_{\mu\nu}=\partial_{\mu}V_{\nu}-\partial_{\nu}V_{\mu}$, thus omitting the “non-Abelian” part, which would contribute to interactions with three and four gauge bosons that we ignore here. $A$ is a model dependent dimensionless anomaly coefficient: $1\lesssim A\lesssim 10$. For instance, in the model analysed in Bizot:2018tds $A$ is given by the dimension of the representation of the hyper-fermions. Note that there are no couplings of type $SSV$ since the $S$ does not acquire a VEV. Also, there is no anomalous coupling $SF_{\mu\nu}\widetilde{F}^{\mu\nu}$ to the EM field, thus the decay $S\to\gamma\,\gamma$ is highly suppressed and for $m_{S}\lesssim 2m_{W}$ the decay $S\to Z\,\gamma$ has near 100% branching ratio. Once again, we can match the current model with the remaining couplings of the phenomenological Lagrangian in eq. (1):

[TABLE]

The mass of $S$ is expected to be small $m_{S}\lesssim m_{h}$ and thus in the region where the decay into $Z\,\gamma$ is motivated. In this particular model, it is given by $m_{h}/(2\cos\theta)$ plus corrections proportional to explicit underlying fermions masses, which are disfavoured by fine tuning arguments. For $t_{R}$ symmetric for example, $m_{\eta}$ tends to vanish and should get its mass completely from underlying fermion masses. Other representations and other models give different expressions, but all agree on the approximate estimate that $m_{S}$ is light due to its pNGB nature.

As far as direct $S$ production goes, we observe that, choosing the spurion embeddings as above, no diagonal coupling of type $S\,\bar{t}^{i}t^{i}$ ($t^{i}=t,\,t^{\prime},\,t^{\prime\prime},\,t^{\prime\prime\prime}$) is directly generated Gripaios:2009pe . This means that the gluon fusion process is not present and the direct production proceeds mainly via EW vector bosons. Diagonal fermionic couplings for the top and for lighter fermions can be induced by further enlarging the model but we ignore them and consider the fermiophobic case. The coupling of $S$ to fermions is nevertheless generated via loop of gauge bosons and might be relevant for low $m_{S}$ Bauer:2017ris ; Craig:2018kne .

3 LHC constraints from $\gamma\gamma$ and $Z\gamma$ resonance searches

To perform a phenomenological analysis of the $\gamma\gamma$ and $Z\,\gamma$ final states it is necessary to estimate the allowed regions in the masses of the VLQ and (pseudo)scalar. This is done in this section by recasting one ATLAS and one CMS search at ${13\,\mathrm{\text{Te\kern-0.43057ptV}}}$ and providing the ensuing limits in the $m_{t^{\prime}}$ vs $m_{S}$ plane.

The searches used for the recast are briefly described in the following.

• •

An ATLAS “Search for new phenomena in high-mass diphoton final states” Aaboud:2017yyg , used to set constraints for the $\gamma\gamma$ final state. This search looks for resonances with spin 0 or 2 decaying into two photons. For the spin 0 resonances (of interest for our analysis) the explored diphoton invariant mass region ranges from ${200\,\mathrm{\text{Ge\kern-0.6458ptV}}}$ to ${2700\,\mathrm{\text{Ge\kern-0.6458ptV}}}$. The search cuts on the transverse energy of the leading and subleading identified photons, $E_{\text{T}}>40\,\text{Ge\kern-0.6458ptV}$ and $E_{\text{T}}>30\,\text{Ge\kern-0.6458ptV}$, respectively, and requires $E_{\text{T}}$ to be larger than a fraction of the diphoton invariant mass, $E_{\text{T}}>0.4m_{\gamma\gamma}\,\text{Ge\kern-0.6458ptV}$ (leading photon) and $E_{\text{T}}>0.3m_{\gamma\gamma}\,\text{Ge\kern-0.6458ptV}$ (subleading photon).

• •

A CMS “Search for standard model production of four top quarks with same-sign and multilepton final states” Sirunyan:2017roi , used to set constraints for the $Z\,\gamma$ final state. This search looks for final states with two (same-charge) or three leptons, and different numbers of jets and $b$-jets, depending on the signal region. No cuts are imposed on photons in the final state. The most relevant cuts are applied to the jet and $b$-jet multiplicity and differ depending on the signal region.

The recast simulations are done using MadGraph5_aMC@NLO madgraph with a dedicated UFO Degrande:2011ua model file corresponding to the simplified Lagrangian in eq. (1). Events are generated at leading order and interfaced with Pythia 8.2 pythia and Delphes 3 delphes for showering and fast detector simulation. As Parton Distribution Functions (PDFs), the NNPDF 3.1 at NLO set Ball:2017nwa has been chosen, obtained through the LHAPDF 6 library Buckley:2014ana using PDF ID 303400. The recast and validation of the searches is then performed through MadAnalysis 5 Conte:2012fm ; Conte:2018vmg .

Simulations have been performed in a grid of $t^{\prime}$ and $S$ masses: $m_{t^{\prime}}$ has been varied in the range ${400\,\mathrm{\text{Ge\kern-0.6458ptV}}}$ to ${1000\,\mathrm{\text{Ge\kern-0.6458ptV}}}$ in steps of ${100\,\mathrm{\text{Ge\kern-0.6458ptV}}}$, while $m_{S}$ starts from a minimum value of ${200\,\mathrm{\text{Ge\kern-0.6458ptV}}}$ and increases in steps of ${100\,\mathrm{\text{Ge\kern-0.6458ptV}}}$ until reaching the kinematical limit $m_{t^{\prime}}-m_{S}-m_{t}=0$. A point in the small mass gap region $m_{t^{\prime}}-m_{S}-m_{t}=10\,\text{Ge\kern-0.6458ptV}$ has been included as well.

The results are shown in figure 2 as upper limits on the cross section (in pb). The observed bound on the $t^{\prime}$ and $S$ masses, represented as a solid black contour, has been obtained by comparing the upper bounds on the cross section with the cross section for pair production of $t^{\prime}$ obtained at NLO+NNLL through Hathor Aliev:2010zk , under the assumption of 100% $\mathrm{BR}$ for $t^{\prime}\to S\,t$ and for $S\to\gamma\,\gamma$ (figure 2 left panel) or $Z\,\gamma$ (figure 2 right panel) in the narrow width approximation (NWA). The range of validity of the NWA in terms of the ratio between the total width and mass of $t^{\prime}$ is discussed in appendix B. In the $\gamma\,\gamma$ channels the allowed region for $m_{t^{\prime}}$ is above $\sim 600\,\text{Ge\kern-0.6458ptV}$ almost independently of $m_{S}$. In the $Z\,\gamma$ channel the bounds are slightly more sensitive to the mass gap between the VLQ and the (pseudo)scalar, barring statistical fluctuations: the bound on $m_{t^{\prime}}$ is however between $\sim 700\,\text{Ge\kern-0.6458ptV}$ and $\sim 800\,\text{Ge\kern-0.6458ptV}$ for all the allowed $m_{S}$.

The bounds obtained are typically weak compared to dedicated VLQ searches. We stress, however, that the bounds provided in this section are simply meant to give an idea about the optimal sensitivity of current searches for the final states considered above. In realistic scenarios the BRs of $t^{\prime}$ and $S$ into such final states will be likely smaller than 100%, which trivially implies that the bounds will get weaker. In this case, other channels might be more sensitive depending on the BRs of the $t^{\prime}$ (and the recasting of different searches more sensitive to other final states has been performed, e.g. in Cacciapaglia:2019zmj , after the appearance of this analysis). Indeed, only a combination of bounds from different final states would give a full picture for any given benchmark point (defined in terms of masses and BRs of $t^{\prime}$ and $S$). The way bounds are provided in figure 2, however, represents one of the elements of this picture. As a practical example, if a benchmark is considered in which the BRs of $t^{\prime}\to S\,t$ or $S\to\gamma\,\gamma$ or $Z\,\gamma$ are smaller than 100%, the observed upper limits on the cross section represented by the grid of numbers in figure 2 can be directly compared with the $\sigma\times$ BRs of a given benchmark to determine the corresponding bound.

In the next section we propose a dedicated analysis to look for the signatures we are interested in leading to a much better sensitivity than the ones presented in figure 2.

4 Analysis

In its full generality, a top partner $t^{\prime}$ may decay into the usual three SM channels $W^{+}\,b$, $Z\,t$, $h\,t$ or additional exotic channels. In this paper we are focusing our attention on the case of pair production $p\,p\to t^{\prime}\,\bar{t}^{\prime}$ and subsequent decay into the BSM channels $t^{\prime}\to S\,t$, where $S$ is a neutral (pseudo)scalar decaying into SM EW diboson pairs. We have chosen the decays $S\to\gamma\,\gamma$ and $S\to Z\,\gamma$ as our target signal, since they are experimentally very clean bosonic decay channels. In the case of the $Z\,\gamma$ channel we only consider further leptonic decays of the $Z$.

The analyses are optimised to look for only one pair of photons or $Z\gamma$ final states originating from the same $S$. When limits from these analyses are reinterpreted in specific models, the BRs of the $S$ can significantly affect the limits therein. In order to reinterpret the results in the models described in section 2, we need to evaluate the efficiencies of the signal region cuts while taking into consideration all possible decays of $S$. We assume $t^{\prime}$ decays at 100% rate as $t^{\prime}\to S\,t$. For $S$, we consider all the possible bosonic decay channels necessary to ensure gauge invariance in the CHM,444Note that additional sizable decays are present for the 2HDM+VLQ case, specifically, $gg$, $t\bar{t}^{(*)}$ and $hh$ (as appropriate for $S=H$ and $A$) decays, which are then simulated or estimated for the corresponding signal.

[TABLE]

In this section we briefly define the objects used in the analyses (with a longer discussion for reproducibility in appendix C), then describe the tools and processes for the simulation of events to model signal and background (section 4.2), and finally we present event selections to extract the signal in the two considered signal regions (SR): the $\gamma\,\gamma$ SR in section 4.3 and the $Z\,\gamma$ SR in section 4.4.

4.1 Object definition

In the following the definition and selection of objects at reconstructed level are briefly outlined. A more detailed account can be found in appendix C. The default ATLAS Delphes card delphes is used, with minor modifications and calorimeter objects that fall in the calorimeter transition region $1.37<|\eta|<1.53$ are excluded. Isolation and overlap removal is done in the Delphes card for most of the objects.

The basic objects used are photons ($\gamma$), leptons ($\ell$), jets ($j$) and $b$-jets ($j_{b}$). Photons are required to have a $p_{\text{T}}>30\,\text{Ge\kern-0.6458ptV}$ and $|\eta|<2.37$. Leptons in this paper are understood to mean electrons or muons only, and not $\tau$-leptons. Leptons must fulfil $p_{\text{T}}>25\,\text{Ge\kern-0.6458ptV}$ and $|\eta|<2.47$. Jets are reconstructed by using the FastJet fastjet package and Delphes with the anti-$k_{t}$ algorithm Cacciari:2008gp using $R=0.4$. Jets are required to pass $p_{\text{T}}>25\,\text{Ge\kern-0.6458ptV}$ and $|\eta|<2.47$. In Delphes, a $b$-jets is a jet which contains a truth $b$-quark.

The compound objects used are $Z$ bosons, missing transverse energy ($E_{\text{T}}^{\text{miss}}$) and the scalar transverse energy ($H_{\text{T}}$). $Z$ bosons are identified as two opposite-sign same-flavour leptons with $|M_{\ell^{+}\ell^{-}}-m_{Z}|<10\,\text{Ge\kern-0.6458ptV}$, where $M_{\ell^{+}\ell^{-}}$ is the invariant mass of the reconstructed leptons. $E_{\text{T}}^{\text{miss}}$ is defined as $\vec{E}_{\text{T}}^{\text{miss}}=-\sum_{i}\vec{p_{\text{T}}}(i)$ delphes , where $i$ runs over the energy deposits in the calorimeter. $H_{\text{T}}$ is the scalar sum of the $p_{\text{T}}$ of all reconstructed basic objects used in the analysis (jets, muons, electrons and photons).

4.2 Simulations

All simulations in this study have been performed using the following framework: MadGraph 5_aMC@NLO madgraph was used to generate events at leading order accuracy. Pythia 8.2 pythia and Delphes 3 delphes have been used for showering and fast detector simulation, respectively. For the signal simulations, the parton distribution function (PDF) NNPDF 3.1 at NLO set Ball:2017nwa set has been chosen, obtained through the LHPDF 6 library Buckley:2014ana using PDF ID 303400. For the background simulations instead the MadGraph default NNPDF 2.3 LO with PDF ID 230000 has been used.

The numerical values of the pair production cross-sections, which only depend on $m_{t^{\prime}}$, are shown in figure 4. They were computed through Hathor Aliev:2010zk , with NNLO MSTW2008 Martin:2009iq PDFs.

The background of the $\gamma\,\gamma$ SR is dominated by $pp\to\gamma\,\gamma+\text{jets}$ mediated by QCD interactions. The backgrounds $\gamma\,\gamma+t+\text{jets}$ and $\gamma\,\gamma+t\bar{t}$ were found to be negligible and hence are not considered for the diphoton analysis. Events from the $pp\to\gamma\,\gamma+\text{jets}$ process are generated with up to three jets, including jets initiated by $b$-quarks, in the matrix element. The final jets after showering and jet clustering are matched to the original partons with the MLM method Mangano:2006rw as implemented in Pythia. In the simulation of the initial state $b$-quarks are explicitly considered as part of the incoming protons. This accounts for processes with an odd number of $b$-jets in the final state, such as those initiated by $gb\to\gamma\,\gamma+u\bar{u}b$. To ensure enough statistics in the high mass tail the events are generated in slices of the diphoton invariant mass $M^{\mathrm{bkg}}_{\gamma\gamma}$ with $\sim 1$ M events per slice, where $M^{\mathrm{bkg}}_{\gamma\gamma}$ refers to the invariant mass of the generated (not reconstructed) photons. Table 1 lists the slices along with the fiducial cross section for each slice. The invariant mass of the two photons for all slices is shown in figure 5. If there are more than two photons in the event, the pair with invariant mass closer to $160\,\text{Ge\kern-0.6458ptV}$ is shown in this figure. The high-mass slices have small tails towards lower masses, which occurs when one or both of the hard photons is lost in the reconstruction and the selected photons originate from e.g. the hadronisation process. The contribution from these mis-reconstructions is typically small and can be mitigated further with $\Delta R$ cuts on the photons. The small peak at $160\,\text{Ge\kern-0.6458ptV}$ is due to the selection requirement that the invariant mass of the photons is close to $160\,\text{Ge\kern-0.6458ptV}$. The total fiducial cross section in the $M^{\mathrm{bkg}}_{\gamma\gamma}>50\,\text{Ge\kern-0.6458ptV}$ region is calculated by generating $25$K events in the allowed range using the same setup as in the full event generation, resulting in ${74.0\,\mathrm{pb}}$, in good agreement with the sum of the fiducial cross sections for the individual slices.

The dominant background in the $S\to Z\,\gamma$ final state is $pp\to Z\,\gamma+\text{jets}$, with $Z\to\ell^{+}\ell^{-}$. Events from this process are generated using the same setup as for the $\gamma\,\gamma+\text{jets}$ background, with up to two hard jets in the matrix elements. For the same reason as for $\gamma\,\gamma+\text{jets}$ the event generation for the $Z\gamma+\text{jets}$ background is performed in slices of the invariant mass of the generator-level $Z$ and $\gamma$, $M^{\mathrm{bkg}}_{Z\gamma}$, with $\sim 2$M events each, listed in table 1 together with their fiducial cross section. The latter at $M^{\mathrm{bkg}}_{Z\gamma}>50\,\text{Ge\kern-0.6458ptV}$ is estimated to be ${4.451\,\mathrm{pb}}$ by generating $25$K events in the allowed kinematic range, which, again, is in good agreement with the sum of the fiducial cross sections of the slices. SM top-quark pair production associated to a photon and to a $Z$ and a photon can also give relevant contributions to the background. We generated 150K events of the process $t\bar{t}+Z\gamma$ and let the top decay inclusively and the $Z$ leptonically via MadSpin. For $t\bar{t}+\gamma$ we generated 300K events and required the top quarks to decay leptonically to either electrons or muons. We use the LO cross sections ${0.315\,\mathrm{fb}}$ for decayed $t\bar{t}+Z+\gamma$ and ${94\,\mathrm{fb}}$ for decayed $t\bar{t}+\gamma$ events. The invariant mass of the $Z\,\gamma$ system, for each of the mass slices of $Z\,\gamma$ + jets, together with $t\bar{t}+\gamma$ and $t\bar{t}+Z+\gamma$, is shown in figure 6. In that figure, at least one $Z$ boson, one photon and one $b$-jet, according to the definitions in section 4.1, are required. If there are more than one $Z$ and/or $\gamma$ candidate we choose the system with invariant mass closer to $160\,\text{Ge\kern-0.6458ptV}$ to present in this specific plot.

In both final states, non-prompt backgrounds are also possible. These are expected to be reduced significantly since we use tight identification requirements for leptons and photons. Furthermore, in analyses with similar final states, the backgrounds with one or more jets mis-identified as photons was found to be significantly smaller than those with prompt photons Aaboud:2017yyg_aux . Thus, we do not consider non-prompt background sources in either of the final states.

For the signal simulation and definition, we generated the process $pp\to t^{\prime}\bar{t^{\prime}}$ with $t^{\prime}\to S\,t$ and $S$ decaying into EW bosons, eq. (20). We define our signal samples as any possible decay combination, $(S\to X)(S\to Y)$ where $X,Y\in\{\gamma\,\gamma,Z\,\gamma,WW,ZZ\}$. Both the $Z$ and $W$ decay inclusively in our signal definition.

The UFO model for signal simulations is the same one used for recasting LHC bounds, corresponding to the simplified Lagrangian of eq. (1). Decays of interest are thus turned on or off by setting the corresponding couplings. In the following analysis, couplings are set such that the widths for the top partner $t^{\prime}$ and scalar $S$ are $0.1\%$ of their mass, to allow the use of the NWA. A quantitative determination of this parameter, performed in appendix B, is essential to determine the range of validity of signal simulations in experimental analyses and also for the subsequent reinterpretation of results in terms of theoretical models.

For the simulations, we use $\kappa_{S}^{R}=0$, keeping only the $\kappa_{S}^{L}$ coupling. This is an important assumption, as fixing a different chirality of the top coupling can lead to observable differences. Indeed, it is known that the dominant chirality of the couplings of a VLQ interacting with the SM top quark can be probed by looking at the transverse momentum of the decay products of the $W$ boson emerging from the top quark Belanger:2012tm ; Barducci:2017xtw . Differently from the SM case, however, here the kinematics of the decay products of $t^{\prime}$ is not only affected by its mass, but also by the $S$ mass.

Similarly we turn off the scalar $S$ couplings, $\kappa_{W}=\kappa_{B}=\lambda_{W}=\lambda_{Z}=0$, when we assume a pseudoscalar nature of the $S$ state. The scalar or pseudoscalar nature of $S$ can also in principle affect the kinematical distributions of its decay products. We have therefore performed simulations imposing specific decay channels, to check, at reconstruction level but without including detector effects, how large differences can be between the above scenarios in differential distributions. We found that there is no observable difference in our predictions with respect to a scalar $S$ in terms of kinematical distributions. In view of this indistinguishability, in the 2HDM+VLQ case, we will assume the $S$ state to represent alternatively a CP-even and CP-odd neutral Higgs states entering the $t^{\prime}$ decay.

4.3 $S\to\gamma\gamma$ signal region

In this section, the diphoton final state is presented. From an experimental point of view, the diphoton final state gives a very clean signature in the detector, which makes it attractive to study.

We considered $t^{\prime}$ masses $m_{t^{\prime}}=600$ to $1800\,\text{Ge\kern-0.6458ptV}$ in steps of $200\,\text{Ge\kern-0.6458ptV}$, every kinematically allowed $S$ mass is investigated, via the discrete values of $m_{S}=100\,\text{Ge\kern-0.6458ptV}$, $200\,\text{Ge\kern-0.6458ptV}$, $400\,\text{Ge\kern-0.6458ptV}$, and then in steps of $200\,\text{Ge\kern-0.6458ptV}$ up to the highest kinematically available mass, $m_{S}=m_{t^{\prime}}-200\,\text{Ge\kern-0.6458ptV}$. The wide selection of $S$ and $t^{\prime}$ masses enables the possibility to study both threshold effects and highly boosted decay products.

To select the signal we demand the presence of 2 photons and 1 $b$-jet defined according to section 4.1. If more than one pair of photons is present we choose the pair whose invariant mass is closer to $m_{S}$ and define these photons as “best” photon candidates, $\gamma_{1}$, $\gamma_{2}$. Unless otherwise specified, a pair of photons is assumed to be the “best” pair. The invariant mass of the system with the two “best” photon candidates is required to be within $20\,\text{Ge\kern-0.6458ptV}$ from the nominal $S$ mass, $|M_{\gamma\gamma}-m_{S}|<20\,\text{Ge\kern-0.6458ptV}$.

In order to further enhance the signal discrimination with respect to the background for low $m_{S}$ values we use the fact that the $S$ is produced in a boosted regime. The top partners $t^{\prime}$ and $\bar{t^{\prime}}$ will be produced nearly at rest and the pair will be back-to-back. The large difference in mass between $t^{\prime}$ and $S$ will make $S$ boosted and thus also the photon pair from $S$ will be collimated. In figure 7 we show the $\Delta R_{\gamma\gamma}$ distributions for different $m_{S}$ and for $m_{t^{\prime}}=800\,\text{Ge\kern-0.6458ptV}$ fixed. We take advantage of this characteristic signal profile and require $\Delta R_{\gamma\gamma}<2.3$ from $m_{S}=100\,\text{Ge\kern-0.6458ptV}$ to $m_{S}=200\,\text{Ge\kern-0.6458ptV}$.

The selection cuts are summarised in table 2. Note that, due to limitations in statistics, the cuts are sub-optimal. The discrimination between signal and background could be improved significantly by tightening the cuts in a real experimental analysis.

In table 3 we show the efficiencies (number of events left after the cut divided by the number before the cut) of the selection cuts numbered in table 2 for different $m_{S}$ values. In the upper part of the table, the signal process is defined with both $S$ decaying into diphotons, i.e., $t\bar{t}S(\to\gamma\,\gamma)S(\to\gamma\,\gamma)$ in the final state. This is the process we use to optimise the selection cuts. We display only the $m_{t^{\prime}}=1\,\text{Te\kern-0.43057ptV}$ case in the table. In the lower part of the table, the efficiencies for the background sample are displayed. It can be noticed that the last two cuts are the most efficient ones in removing the background and keeping signal events.

The final efficiencies for the signal decay channel $S(\to\gamma\,\gamma)S(\to\gamma\,\gamma)$ are discussed in section 4.5. The efficiencies for the other signal decay channels with at least one branch decaying into $\gamma\,\gamma$ are presented in appendix D.

4.4 $S\to Z\,\gamma$ signal region

In the $S\to Z\,\gamma$ final state we require at least one $Z$ boson candidate reconstructed according to the definitions in section 4.1. In addition to the $Z$ candidate we require the presence of at least one isolated photon. The system of one isolated photon and one $Z$ candidate whose invariant mass is closest to the nominal $S$ mass is called the “best $S$ candidate”. To efficiently distinguish the signal from the background we exploit the high multiplicity of objects and high total energy of a typical signal event. We require $H_{\text{T}}+E_{\text{T}}^{\text{miss}}>0.3m_{t^{\prime}}$, where $H_{\text{T}}$ is the scalar sum of the $p_{\text{T}}$ of all reconstructed basic objects and $E_{\text{T}}^{\text{miss}}$ is the missing transverse energy of the event as described in section 4.1. We finally require the invariant mass of the $S$ candidate to be within $15\,\text{Ge\kern-0.6458ptV}$ of the nominal $S$ mass, i.e., $|M_{Z\gamma}-m_{S}|<15\text{Ge\kern-0.6458ptV}$. A summary of these selection cuts is presented in table 4, with some information on the object definitions for convenience.

The distributions of $M_{Z\gamma}$ before cut 5 and $H_{\text{T}}+E_{\text{T}}^{\text{miss}}$ before cut 4 and 5 are shown in figure 8, for the masses $m_{S}=160\,\text{Ge\kern-0.6458ptV}$ and $m_{t^{\prime}}=1400\,\text{Ge\kern-0.6458ptV}$. There is a great discriminating power in the $H_{\text{T}}+E_{\text{T}}^{\text{miss}}$ observable due to the large multiplicity and energy of a typical signal event. We note that the used cut is not optimised to suppress the background due to lack of MC statistics. A realistic experimental analysis could harden this cut to further reduce the background and use data-driven methods to estimate it without relying too much on MC estimates.

For illustrative purposes, in table 5, we show the efficiencies of the selection cuts numbered in table 4 for different $m_{S}$ values. We display only the case $m_{t^{\prime}}=1400\,\text{Ge\kern-0.6458ptV}$ in the table. In the upper subtable, the signal process is defined with both $S$ decaying into $Z\,\gamma$, $S(\to Z\,\gamma)S(\to Z\,\gamma)$ in the final state. This is the process we use to optimise the selection cuts. In the lower subtable, the efficiencies for the background sample are displayed. Except the mass-window cut for the $S$ candidates, all cuts depend on $m_{t^{\prime}}$.

4.5 Efficiencies

The signal efficiencies for the two different signal regions are the last piece of information necessary for reconstructing the number of signal events. In figure 9 we provide, as illustrative examples, the efficiencies for the $(\gamma\,\gamma)(\gamma\,\gamma)$ channel in the $\gamma\,\gamma$ SR and for the $(Z\,\gamma)(Z\,\gamma)$ channel in the $Z\,\gamma$ SR, for which the selections have been optimised. Further efficiency plots for different channels are provided in appendix D. All efficiencies have been computed considering signal samples of $10^{4}$ MC events, corresponding to a statistical uncertainty of the order of 10% which can affect the evaluation of efficiencies especially when they are small. The whole set of efficiencies, combined with the $\mathrm{BRs}$ chosen in section 4, allows one to compute the expected total number of events via eq. (22) in the following section, where the results of the study are discussed.

In the next section we will show how to estimate the number of events for both signal and backgrounds for different model assumptions and devise a simple statistical framework for model interpretation.

5 Results

In this section we discuss the discovery potential of LHC for the models introduced previously. Essentially, we propose a counting experiment comparing the number of expected background events with the number of signal events.

The expected number of background events in one of the signal regions SR $\in\{\gamma\,\gamma,Z\,\gamma\}$, $B_{\rm SR}$, is given by

[TABLE]

with $L$ the integrated luminosity, and $\sigma_{B_{\gamma\,\gamma}}=74.0\,\text{pb}$ and $\sigma_{B_{Z\,\gamma}}=4.58\,\text{pb}$ our best estimate of the total background cross section for the $\gamma\,\gamma$ and $Z\,\gamma$ signal regions, respectively, and $\epsilon_{B_{\rm SR}}$ the efficiency after all cuts in the corresponding SR.

The number of background events can be extracted for arbitrary values of $m_{S}$ and $m_{t^{\prime}}$ by interpolating the data presented in tables 6–7.

It should be noted that we only present the estimates for the irreducible background. This turns out to be negligible in the high mass region and its values are presented only to show this fact and for completeness. Fake rates are also expected to be negligible in the high-mass region ATL-PHYS-PUB-2013-009 .

The number of expected signal events for each SR is given by

[TABLE]

where $\epsilon_{\rm SR}^{Y,X}$ is the final efficiency in appropriate signal region SR for the signal sample with decay $(S\to X)(S\to Y)$ with $X,Y\in\{\gamma\,\gamma,Z\,\gamma,WW,ZZ\}$. (In these expressions we assume the validity of the NWA and assume 100% $\mathrm{BR}$ $t^{\prime}\to S\,t$ and $\bar{t}^{\prime}\to S\,\bar{t}$.)

In appendix D we tabulate the above efficiencies, allowing one to estimate the signal in any of the theoretical models discussed here by simply computing the corresponding $\mathrm{BR}$. The discovery potential for a more generic model can be also estimated using the numbers provided as long as the efficiency times BR of any extra decay channel is known to be small.

Having computed the number of signal ($S$) and background ($B$) events, we estimate the significance by employing the formula Li:1983fv ; Cousins:2007bmb ; Cowan:2010js

[TABLE]

that is obtained by using the “Asimov” data-set into the profile likelihood ratio. The explicit expression above, containing the uncertainty $\sigma_{b}$ on the background, is found in ref. CowanMPI .

We consider an overall $\sigma_{b}=10\%B$ systematic uncertainty on $B$. This number is most likely a conservative estimate and it is estimated by comparing the systematic uncertainties of ATLAS and CMS analyses with similar final states, especially high-mass $Z\,\gamma$ searches Aaboud:2017uhw ; Sirunyan:2017hsb and high mass $\gamma\,\gamma$ searches Aaboud:2018ewm ; Aaboud:2018ftw ; Sirunyan:2018wnk .

5.1 Model interpretation

Recall that the main focus is the study of models where the top partner has 100% BSM $\mathrm{BR}$ $t^{\prime}\to S\,t$ and $S$ decays into EW gauge bosons. Even within this limited framework, we still need to discuss the relative strengths of the various $S$ decay channels, controlled by the couplings in eq. (1).

We start by considering the optimal reaches for the two SR considered in this analysis, corresponding to scenarios where $S$ decays fully either into $\gamma\,\gamma$ or $Z\,\gamma$. Such scenarios are likely non-physical, but they allow to determine the maximum potential of the selections. The LHC reaches for this simplified scenario are presented in figure 10 for two different LHC luminosities, corresponding to the final luminosity at the end of Run II and the nominal final luminosity of Run III. It can be noticed that the sensitivity of the search diminishes for increasing $m_{t^{\prime}}$ due to the reduction of production cross section, but it improves with increasing $m_{S}$ because of the reduction of the background yields (see table 7).

We now move on to more theoretically motivated scenarios. We first consider the benchmark motivated by partial compositeness, where only the anomaly induced pseudoscalar couplings $\tilde{\kappa}_{B}$ and $\tilde{\kappa}_{W}$ are non-zero.

In this case, the structure of the anomaly coefficients Bizot:2018tds in all explicit realizations gives $\tilde{\kappa}_{B}+\tilde{\kappa}_{W}=0$, thus suppressing the $S\to\gamma\,\gamma$ decay. This leads to a 100% $\mathrm{BR}({S\to Z\,\gamma})$ below the $WW$ threshold and still an acceptably large value above it, as displayed in figure 11 (left). The LHC reaches for this scenario are presented in figure 11 (right) for two different LHC luminosities, corresponding to the final luminosity at the end of Run II and the nominal final luminosity of Run III. Here, we consider only the $Z\,\gamma$ SR because of the negligible sensitivity of the $\gamma\,\gamma$ SR.

Different effects are present in the reach of figure 11. For $m_{S}\lesssim 2m_{W}$ the sensitivity is optimal due to a 100% decay rate of both $S$ into $Z\,\gamma$ ($S\to Z\,\gamma,\,S\to Z\,\gamma$) and a high efficiency (figure 9 (right)). Above threshold the $S\to VV,\,S\to VV$ ($V=W,Z$) decay channels kick in with $\approx 64\%$ rate and negligibible efficiency, while the $S\to Z\,\gamma,\,S\to Z\,\gamma$ rate reduces to $\approx 4\%$. The mixed decay $S\to VV,S\to Z\,\gamma$ takes 16% of the branching ratio and have an efficiency approximately constant and near 40% compared to the pure $Z\,\gamma$ case (figure 21). This depletion in the signal explains the kink of sensitivity lost near the $m_{S}\approx 2m_{W}$ threshold. In both regions the sensitivity improves with increasing values of $m_{S}$ due to a rapid decrease of the background, as noticed in figure 10.

The interpretation for the composite Higgs model described in section 2.3 is straightforward. The $S$ is photophobic and we can read the bounds directly from figure 11. It is encouraging to see that even for not optimised cuts this channel could be competitive with the search for the $+5/3$ charged partner Sirunyan:2018yun . Some more details for this model are given in appendix A.

For the 2HDM+VLQ case, the interpretation is somewhat more complicated because of the more numerous parameters, richer particle spectrum and, hence, the decay patterns of $t^{\prime}$ and $S$. A scan has been performed by varying the 2HDM input parameters described in appendix A.1 to obtain benchmark points characterised by the highest BRs of $t^{\prime}$ and $S$ into the final states considered in this analysis in order to maximise the sensitivity. Such points are simply representative of the 2HDM spectrum, as we ignored the fact that HiggsBounds excludes the majority of them. In fact, the scope of this selection is to illustrate the potential of the model independent analysis developed in this paper rather than to constrain specific theoretical models. We first restricted the scan by enforcing an almost exclusive decay of the $t^{\prime}$ into the CP-even scalar $H$ by setting the masses of the CP-odd $A$ and charged $H^{\pm}$ states to high values and by restricting the 2HDM input parameters in such a way that SM decays of the $t^{\prime}$ are also suppressed. We then computed the BRs of $t^{\prime}$ and $H$ as a binned function of their masses by considering the median of the sample for each bin. This procedure approximates the BRs neglecting any correlation point-by-point and is reasonably accurate given the size of the sample (approximately 30,000 points). In fact, we have verified that the sum of the BR functions obtained with this procedure is approximately 1 for all $t^{\prime}$ and $H$ masses. Examples of the distribution of scanned points and of the median BRs are provided in figure 12 for $t^{\prime}\to Ht$ and $H\to\gamma\,\gamma$. (The procedure is identical for the case of $t^{\prime}\to At$ and $A\to\gamma\,\gamma$, though the point distributions and median values are obviously different.)

The decay of $H$ into $\gamma\,\gamma$ is around 0.3% below the $hh$ threshold, while its decay into $Z\,\gamma$ is $\sim$0.05%. The generically dominant decay of $H$ is into $gg$, which is on average around 70%, followed by $WW$ ($\sim$20%) and $ZZ$ (5% to 10%), while the BRs into $b\bar{b}$ and $c\bar{c}$ are 1% or less. Above the $2m_{h}\approx 250\,\text{Ge\kern-0.6458ptV}$ threshold, $H\to hh$ dominates and all other BRs drop significantly, until the (on-shell) $t\bar{t}$ channel opens and becomes dominant. Then, we do a second scan with the role of $H$ and $A$ interchanged (approximately 80,000 points) and compute the BRs as described above. Here, there cannot be $WW$ and $ZZ$ decays of the $A$ state, so that $gg$ and $b\bar{b}$ decays share the majority oft the decay rate (about 90% of it, with the remainder saturated by $\tau^{+}\tau^{-}$ and $Zh$, which we then neglect in the MC generation) till the (off-shell) $t\bar{t}^{*}$ channel opens (coincidentally enough, around $m_{t}+m_{b}+m_{W}\approx 260\,\text{Ge\kern-0.6458ptV}$), with the $\gamma\,\gamma$ and $Z\,\gamma$ rates being generally lower than in the previous case. Given the low BR into $Z\,\gamma$, in addition to the subleading BR of the $Z$ into leptons, no significant sensitivity is expected in the $Z(\to\ell^{+}\ell^{-}))\gamma$ final state and, therefore, we will focus only on the $\gamma\,\gamma$ SR for the 2HDM+VLQ in the case of both a light $H$ and $A$.

The efficiencies for $\{\gamma\,\gamma,WW\}$ and $\{\gamma\,\gamma,ZZ\}$ are provided in appendix D. Given the high BR into $gg$, the efficiencies have been computed for the $\{\gamma\,\gamma,gg\}$ final state as well. This has been done only in the region of parameter space where high sensitivity is obtained, i.e., for $m_{t^{\prime}}$ less than 1 TeV, and are in average around 20%. The efficiencies for the $\{\gamma\,\gamma,hh\}$ channel have also been calculated above the $H\to 2h$ threshold, in the region of high sensitivity, and are found to be around 30%. Given the illustrative nature of this example, we assumed the efficiencies for the $\{\gamma\,\gamma,t\bar{t}^{*}\}$ in the case of a light $A$ channel to be flat and around 30%.

The results for the 2HDM+VLQ are shown in figure 13. For the case of a light $H$ state, some discovery reach has been found for $m_{t^{\prime}}$ around 600 GeV and exclusion is possible up to $m_{t^{\prime}}$ around 700 GeV, almost independently of $m_{S}$ below the (on-shell) $t\bar{t}$ threshold. For the case of a light $A$ state, the reach in $m_{t^{\prime}}$ for both discovery and exclusion is somewhat deeper than in the previous case, by some 50 GeV. In contrast, the one in $m_{S}$ is very similar, as it again collapses at approximately $2m_{t}$.555Note that, while one may want to consider the case of both $H$ and $A$ being light (and possibly degenerate in mass) in order to benefit from an increased $S$ signal rate, this is impractical because, on the one hand, the $H^{\pm}$ boson also ought to be light to preserve EWPT compliance (thereby increasing the $t^{\prime}\to H^{\pm}b$ BR) and, on the other hand, the impact of the restrictions from HiggsBounds  increases substantially.

6 Conclusions

While the case for VLQs, especially those of top flavour, has already been well established from the theoretical side, the experimental pursuit of their signatures at the LHC has been somewhat limited, as ATLAS and CMS analyses have primarily been carried out under the assumption that such new states of matter decay into SM particles only, i.e., via $t^{\prime}\to W^{+}\,b,Z\,t$ and $h\,t$. This approach clearly enables one to make the most in terms of optimising the signal-to-background ratio in an analysis, chiefly because one can attempt reconstructing the measured $W^{+}$, $Z$ and $h$ masses. However, if one considers VLQ models with additional particles this is overly restrictive since the VLQ may decay via exotic channels involving scalars or pseudoscalars. While the kinematic handles available to enhance these exotic channels may be apparently limited in comparison (as the exotic scalar or pseudoscalar states may have not been discovered already and/or their mass not measured), the size of the associated $\mathrm{BRs}$ could be large enough so as to nonetheless enable sensitivity to these channels. Furthermore, if the companion Higgs states are heavier than the $W^{+}$, $Z$ and $h$ objects of the SM, the signal would anyhow be present in a region of space where the background contamination is minimised. Based on this reasoning, in this paper, we have set out to assess the scope of the LHC to test $t^{\prime}$ decays into neutral (pseudo)scalar states, whose nature could be either fundamental or composite. As an example of spin-0 fundamental states, we have assumed here a Higgs sector comprised of the SM state supplemented by a scalar boson as well as a 2HDM (Type-II) containing both a scalar and pseudoscalar state (which we have taken light one at a time). As an example of spin-0 composite states, we have looked at a CHM where an additional pseudoscalar state emerges as a pNGB of the underlying new strong dynamics. In fact, we have also shown how all such models can conveniently be parametrised in the form of a simplified model onto which they can be mapped.

Of the various possible decay modes of this additional neutral (pseudo)scalar bosons, which we have collectively labelled as $S$, we have considered here two of the cleanest probes possible at the LHC, i.e., $S\to\gamma\,\gamma$ and $Z\,\gamma$ (with the $Z$ decaying into electron/muon pairs). In doing so, we have performed a dedicated signal-to-background analysis exploiting parton level event generation, QCD shower and hadronisation effects as well as detector emulation aimed at establishing the sensitivity of the LHC experiments to such decays, where the $S$ state emerges from a companion top decay, $t^{\prime}\to S\,t$, following $t^{\prime}\bar{t}^{\prime}$ production (with the $\bar{t}^{\prime}$ decay treated inclusively). In the case of both $S$ signatures, we have not attempted any reconstruction of the SM top quark entering the $t^{\prime}$ decay chain although, on a trial-and-error basis, we have assumed knowledge of the $S$ mass, to be able to exploit both the cleanliness of the two $S$ decay channels and the ability of a standard LHC detector in sampling $\gamma\,\gamma$ and $Z(\to\ell^{+}\ell^{-})\gamma$ invariant masses with high resolution. Indeed, this approach also enables us to compare on a more equal footing the scope of $t^{\prime}\to S\,t$ signatures with that of $t^{\prime}\to W^{+}\,b,\,Z\,t$ and $h\,t$ ones, where a mass reconstruction is normally imposed on the $W^{\pm}$, $Z$ and $h$ decay products.

As a result of this approach, we have found that the $t^{\prime}\to S\,t$ signatures give a level of sensitivity not dissimilar from that obtained through studies of $t^{\prime}\to W^{+}\,b,\,Z\,t$ and $h\,t$. For specific regions of the parameter space of VLQ models with exotic Higgs states, which have survived all available constraints from both direct and indirect $t^{\prime}$ and $S$ searches (including those obtained by ourselves from recasting experimental studies for other sectors), we have found the following exclusion and discovery reaches. For a simplified model maximising both the $t^{\prime}$ and $S$ $\mathrm{BRs}$, $m_{t^{\prime}}$ can be probed in both the $\gamma\,\gamma$ and $Z\,\gamma$ channels up to approximately ${2\,\mathrm{\text{Te\kern-0.43057ptV}}}$ for $S$ masses well into the TeV region. In the CHM scenario considered, coverage is not dissimilar for the $\gamma\,\gamma$ case but for the $Z\,\gamma$ the $t^{\prime}$ reach is limited to ${1.6\,\mathrm{\text{Te\kern-0.43057ptV}}}$. Finally, in the 2HDM+VLQ, it is possible to exclude $m_{t^{\prime}}$ up to around 700(750) GeV and discover $m_{t^{\prime}}$ up to around 600(650) GeV almost independently of $m_{S}$ when $S$ represents the CP-even(odd) $H(A)$ state and below the (on-shell) $t\bar{t}$ threshold for the decay of $S$. This is limited to the $\gamma\,\gamma$ case, though, as $Z\,\gamma$ gives no sensitivity at both Run II and III.

Hence, in connection to all of the above, we can confidently conclude to have surpassed the state-of-the-art in VLQ searches in two respects: firstly, by testing the scope of non-SM decays of the $t^{\prime}$ state and, secondly, by deploying a selection procedure which is model independent yet enables one to interpret its results in a variety of theoretical scenarios. Furthermore, it should be noted that, while restricting ourselves to the case of $\gamma\gamma$ and $Z\gamma$ signatures of the (pseudo)scalar states emerging from the described VLQ decays, there is no reason why our procedure cannot be applied to other $S$ decays. Indeed, it can also be further improved (e.g., by reconstructing top-quark decays).

In summary, we believe that there is significant margin for improving the sensitivity of the LHC to models with a heavy top partner, through the exploitation of its decay channels into exotic (i.e., non-SM-like) neutral (pseudo)scalar states, which are ubiquitous in BSM constructs containing such a new fermion. In fact, over sizeable regions of the parameter space of the realistic VLQ models considered here, we have found that sensitivity to both the $t^{\prime}$ and $S$ mass can extend well into the TeV region, thereby being competitive with the currently studied SM channels. While in this paper we have limited ourselves to illustrating this through a few benchmarks examples, in a forthcoming paper, we shall quantify the regions of parameter space of our models where such a phenomenology can be realised, including tensioning the scope of standard and exotic $t^{\prime}$ decays against each other.

Appendix A Details of the models

In this appendix, additional details are given of the models; the 2HDM+VLQ model in appendix A.1 and the composite Higgs model in appendix A.2.

A.1 The 2HDM with an additional VLQ

The scalar potential of the model includes two identical scalar doublets ($\Phi_{1},\Phi_{2})$ and a discrete symmetry $\Phi_{i}\to(-1)^{i}\Phi_{i}$ ($i=1,2$), which is only violated softly by dimension-two terms Branco:2011iw ,

[TABLE]

We take all parameters in the above potential to be real (although $m_{12}^{2}$ and $\lambda_{5}$ could in principle be complex). The two complex scalar doublets may be rotated into a basis where only one doublet acquires a VEV, the Higgs basis,

[TABLE]

where $G^{0}$ and $G^{\pm}$ are the would-be Goldstone bosons and $H^{\pm}$ are a pair of charged Higgs bosons. $A$ is the CP odd pseudoscalar, which does not mix with the other neutral states. The Goldstone bosons are aligned with the VEV in Higgs flavor space, while the $A$ is orthogonal. The physical CP even scalars $h$ and $H$ are mixtures of $\varphi^{0}_{1,2}$ and the scalar mixing is parametrized as

[TABLE]

where $\tan\beta=v_{1}/v_{2}$ is the angle used to rotate $\Phi_{1,2}$ to the Higgs basis fields $H_{1,2}$, $\alpha$ is the additional mixing angle needed to diagonalize the mass matrix of the CP-even scalars, and $s_{\beta-\alpha}=\sin(\beta-\alpha)$, $c_{\beta-\alpha}=\cos(\beta-\alpha)$. The most general renormalisable interaction and mass terms involving the VLQ can be described by the following Lagrangian (where we only include the third generation SM quarks),

[TABLE]

where $\widetilde{H}_{i}\equiv i\sigma_{2}H^{*}_{i}$ ($i=1,2$), $Q_{L}$ is the SM quark doublet and $M$ is a bare mass term for the VLQ, which is unrelated to the Higgs mechanism of EWSB. Note that often the Yukawa couplings of the 2HDM are written in terms of the fields $\Phi_{1},\Phi_{2}$. In eq. (37) we use the Higgs basis fields, so the Yukawa couplings $y_{T},\xi_{T}$ must be defined accordingly. In a Type II-model, as we are considering in this paper, the up-type quarks only couple to the doublet $\Phi_{2}$, while down-type quarks only couple to $\Phi_{1}$. Additional mixing terms of the form $\overline{T}_{L}t_{R}$ can always be rotated away and reabsorbed into the definitions of the Yukawa couplings. In the weak eigenstate basis $(\widetilde{t},T)$, where $\widetilde{t}$ is the SM top quark, the top quark and VLQ mass matrix is

[TABLE]

where $y_{t}$ is the Yukawa coupling of the top quark. It is clear from the above mass matrix that the physical mass of the heavy top, $m_{t^{\prime}}$, is different from $M$ due to the $t$–$T$ mixing. The mass matrix ${\mathcal{M}}$ can be diagonalised by a bi-unitary transformation in the same way as in section 2.1 to obtain the physical states $(t_{L,R},t^{\prime}_{L,R})$ in terms of the gauge eigenstates $(\widetilde{t}_{L,R},T_{L,R})$,

[TABLE]

The mixing angles $\theta_{L}$ and $\theta_{R}$ are not independent parameters. From the bi-unitary transformations we can derive the relations

[TABLE]

and by using the traces and determinants

[TABLE]

we end up with the relations

[TABLE]

and a relationship between $\theta_{L}$ and $\theta_{R}$ and the Yukawa couplings,

[TABLE]

The $t^{\prime}$–$t$ interaction can thus be described by three independent physical parameters: two quark masses $m_{t},m_{t^{\prime}}$ and a mixing angle $s_{L}=\sin\theta_{L}$.

After rotating the weak eigenstates $(\widetilde{t}_{L},T_{L})$ into the mass eigenstates, the Yukawa Lagrangian takes the following form Arhrib:2016rlj :

[TABLE]

where $U_{L,R}$ are the matrices appearing in eq. (49). The neutral Higgs couplings to top ($t$) and top partner ($t^{\prime}$) pairs are in the notation of eq. (1) given by (with $S=H$ or $A$)

[TABLE]

The couplings here are normalised to $y_{t}/\sqrt{2}$, which is what the $H_{\rm SM}t\bar{t}$ coupling would be in the case of no mixing between the $\widetilde{t}$ and $T$ and additionally, in the alignment limit of the 2HDM, $\sin(\beta-\alpha)\to 1$ where the lightest neutral scalar $h$ is the SM-like Higgs boson. Note that in eq. (1) the terms with diagonal couplings $St^{\prime}\bar{t}^{\prime}$ of the top partner to the scalars are not included, since they are not phenomenologically relevant in this paper. We include them in eq. (69) for completeness, however. Note also that the combination $(c_{\beta-\alpha}-s_{\beta-\alpha}\cot\beta)$ that occurs in eq. (69) is proportional to the 2HDM Type II Yukawa coupling of the heavier Higgs boson $H$.

In our analysis we have used a modified version of the public code 2HDMC Eriksson:2010zzb with a VLQ added according to the description above. We have scanned over the parameter space of the model, which is constrained by Higgs data from the LHC that can be evaluated using the public code HiggsBounds Bechtle:2013wla . In addition, 2HDMC can evaluate oblique parameters and theoretical constraints on unitarity, perturbativity and positivity of the potential. However, since our aim here is rather to demonstrate the use of the method developed in this paper, we have not made a comprehensive scan to satisfy these bounds, but instead we have considered parameter points that provide large BRs of $t^{\prime}\to St$ and $S\to\gamma\gamma$ for $S=H$ or $A$. We have therefore chosen to make the Higgs boson that does not play the role of $S$ as well as the charged Higgs boson heavy. We perform random scans over the parameters and generate $10^{5}$ points for each of the scenarios with $S=H,A$. We then keep those points where the product $\mathrm{BR}({t^{\prime}\to S\,t})\times\mathrm{BR}({S\to\gamma\gamma})>10^{-3}$. The scalar $S$ is taken in the range $180\,\text{Ge\kern-0.6458ptV}<m_{S}<350\,\text{Ge\kern-0.6458ptV}$, while for $S=H$ the other heavy scalar is taken in the range $600\,\text{Ge\kern-0.6458ptV}<m_{A}<1000\,\text{Ge\kern-0.6458ptV}$. For $S=A$, instead we choose $m_{H}=1\,\text{Te\kern-0.43057ptV}$. The charged Higgs mass is always $m_{H^{\pm}}=1\,\text{Te\kern-0.43057ptV}$. The remaining Higgs sector parameters are in the ranges $0.99<|s_{\beta-\alpha}|<1$, $0.1<\tan\beta<1$ and we take $m_{12}^{2}=m_{A}^{2}\sin\beta\cos\beta$. Finally, the VLQ couplings are taken in the ranges $500\,\text{Ge\kern-0.6458ptV}<m_{t^{\prime}}<1500\,\text{Ge\kern-0.6458ptV}$, $-0.15<s_{L}<0.15$ and $10<y_{T}<15$.

A.2 The composite Higgs model

As mentioned in the main text, the SM Higgs $\mathcal{H}$ field in this model is a bi-doublet of $\text{SU}(2)_{L}\times\text{SU}(2)_{R}$, which together with a singlet $S$ forms the five dimensional anti-symmetric irrep of $\text{Sp}(4)$,

[TABLE]

The fermionic sector also consists of a bi-doublet and a singlet in the $\mathbf{5}$ of $\text{Sp}(4)$,

[TABLE]

The new fermions mix with the third family quarks of the SM. The mixing is obtained by choosing to embed both the left-handed $Q_{L}=(\widetilde{t}_{L},b_{L})^{T}$ and the right-handed $\widetilde{t}_{R}$ as spurions into the $\mathbf{6}$ of $\text{SU}(4)$. The non-zero components of $Q_{L}$ fit into the bi-doublet of the $\text{SU}(2)_{L}\times\text{SU}(2)_{R}$ subgroup, while $\widetilde{t}_{R}$ is in the singlet of the $\mathbf{6}\to\mathbf{5}+\mathbf{1}$ decomposition of $\text{SU}(4)\to\text{Sp}(4)$. The choice for $Q_{L}$ is essentially dictated by the need to preserve the custodial symmetry of Agashe:2006at .

The construction of the interaction Lagrangian from the general formalism has been addressed in many papers and will not be reviewed here. Suffice it to say that we combine the five pNGBs into a $4\times 4$ matrix $\Pi$ and exponentiate it to obtain

[TABLE]

and use it to “dress” the fermionic field $\Psi$, written as a $4\times 4$ anti-symmetric matrix. In this notation, the Lagrangian becomes

[TABLE]

where we indicated the dressing explicitly. (Note that $Q_{L}\to gQ_{L}g^{T}$ and $\Psi\to h\Psi h^{T}$.)

We allow only the Higgs field to acquire a VEV, and we denote the mixing angle by $\sin\theta=v/f$, where $v=246\,\text{Ge\kern-0.6458ptV}$. Generically $f>800\,\text{Ge\kern-0.6458ptV}$ from EWPT, although one can envisage mechanisms that would allow to lower that bound BuarqueFranzosi:2018eaj .

Computing eq. (73) to all orders in $\theta$ and retaining only terms linear in $h$ and $S$, $h$ being the canonically normalised physical Higgs with VEV shifted to zero, we can write the part of eq. (73) concerning top partners as (see also Bizot:2018tds )

[TABLE]

where the mass and Yukawa matrices are given by

[TABLE]

The singular value decomposition of $\mathcal{M}$ is unwieldy, but can be performed numerically or perturbatively to order $\theta\approx v/f$. For the four top quark mass eigenstates $t,t^{\prime},t^{\prime\prime},t^{\prime\prime\prime}$, the perturbative expressions for the masses are

[TABLE]

The mass of the bottom partner (mostly aligned with $B$) turns out to be of the same order as that of the heaviest top partner $m_{t^{\prime\prime\prime}}$, while $X$ has mass equal to $M\equiv m_{t^{\prime}}$ since it does not mix with anything. For the top quarks, the conversion from gauge to mass eigenbasis reads, to ${\mathcal{O}}\left(v/f\right)$,

[TABLE]

This spectrum justifies that choice of simplified model in the text where we neglect all the top partners other than the lightest one.

Regarding decays of the pseudoscalar in this model, in figure 14 we show the partial widths of $S$ as a function of its mass, including the dominant loop induced fermionic channel $S\to\bar{b}\,b$ relevant below the $Z\,\gamma$ threshold. We use $f/(A\cos\theta)=500\,\text{Ge\kern-0.6458ptV}$ but all curves rescale by $\left(500\,\text{Ge\kern-0.6458ptV}\,A\cos\theta/f\right)^{2}$. We see that for all interesting regions of parameters the width is always very narrow, but still prompt.

The most promising parameter region for this class of models is $m_{S}\lesssim 160\,\text{Ge\kern-0.6458ptV}$, where the $S$ decays dominantly to $Z\,\gamma$. This region is motivated from the model building perspective since it is expected $m_{S}<m_{h}$. From the experimental point of view it offers a clear benchmark of a $Z\,\gamma$ channel. Above $2m_{W}$ the $WW$ channel overcomes, and for $m_{S}\lesssim 80\,\text{Ge\kern-0.6458ptV}$ the $b\bar{b}$ channel dominates, both of which are less clean channels experimentally.

Appendix B Range of validity of the narrow-width approximation

In the processes under consideration both $t^{\prime}$ and $S$ are assumed to be in the narrow-width approximation (NWA), in order to factorise the production of the top partner from its decay chain. Such assumption, however, implies that the coupling $t^{\prime}tS$ cannot exceed specific values which depend on the masses of $t^{\prime}$ and $S$ according to the relation in eq. (2). Considering as a simplifying and extreme assumption that the only available decay channel for $t^{\prime}$ is into the SM top and $S$ and that one chirality of the couplings is dominant with respect to the other, such that either $\kappa^{S}_{R}\ll\kappa^{S}_{L}$ or vice versa, the values of the coupling corresponding to different $\Gamma_{t^{\prime}}/m_{t^{\prime}}$ ratios is shown in figure 15.

For a specific $\{m_{t^{\prime}},m_{S}\}$ configuration, values of the coupling larger than those in the contours of figure 15 would produce a larger width. The determination of the validity of the NWA approximation is important to understand the reliability of the results. If the $t^{\prime}$ width is not narrow, off-shellness effects in the process of pair production and the contribution from topologies which are neglected in the NWA, represented by the examples of figure 16, can become more and more relevant.

To assess how the width of $t^{\prime}$ affects the determination of the cross-section, the full $2\to 4$ process $pp\to t\bar{t}SS$ has been evaluated by imposing the presence of at least one $t^{\prime}$ propagator in the topologies, in order to obtain the signal under the assumption of negligible $Stt$ coupling. With such process, the off-shellness effects and contribution of topologies such as those in figure 16 are fully taken into account. Still under the assumption that $t^{\prime}$ can only decay to $S\,t$ and therefore that the only way to increase the total width of $t^{\prime}$ for a given $\{m_{t^{\prime}},m_{S}\}$ configuration is by increasing $\kappa^{S}_{L,R}$, the ratio between the cross-sections of the full process and of the pair-production process in the NWA is shown in figure 17.

The effect of a large width is already noticeable when the $\Gamma_{t^{\prime}}/m_{t^{\prime}}$ ratio reaches 1%, when the interference between the resonant channels and all the other contributions is negative and of the order of few percents in a region where $m_{S}+m_{t}$ is around 80% of $m_{t^{\prime}}$. If the $\Gamma_{t^{\prime}}/m_{t^{\prime}}$ ratio is below 1% the relative ratio between cross-sections is dominated by the statistical fluctuations of the simulation. For this reason, the numerical results in the following sections assume $\Gamma_{t^{\prime}}/m_{t^{\prime}}$ to be of order 0.1%.

Appendix C Object definition

In the following, more details for the definition and selection of objects at reconstructed level are presented, elaborating on the brief description in section 4.1, in order to facilitate reproducibility and as a guide for possible future searches at colliders.

For all objects, the default ATLAS Delphes card delphes is used, with minor modifications in a few cases, as explained below. Objects that partially fall in the calorimeter transition region $1.37<|\eta|<1.53$ are excluded, if they are reconstructed in the calorimeter, where $\eta$ is the pseudorapidity. Relative angular distances in the detector are typically expressed as $\Delta R$ in the $\eta$-$\phi$ plane where $\phi$ is the azimuthal angle around the beampipe. A particle’s transverse momentum $p_{\text{T}}$ is the momentum component in the plane transverse to the beam axis.

Isolation and overlap removal are needed to distinguish the objects from each other in the detector simulation,666In detectors at colliders, the same energy deposits can be associated with different objects, e.g., an electron can also be identified as a jet. In order to make sure each energy deposit is counted only once, every object has to be energetically isolated and the objects are not allowed to overlap in the detector.

which is done in the Delphes card, unless otherwise specified. This is achieved by creating the containers for the objects in mind: jets, photons, electrons and muons. In Delphes all objects passing their respective efficiency cut are first reconstructed as the respective object and as a jet. The object will then be put into the jet container and the container corresponding to the reconstructed object. By passing an isolation criterion the object is removed from the jet container and only kept in the container corresponding to the correct reconstruction. The criterion is met when an isolation variable $I$ is within a certain constraint. The variable is defined by summing the $p_{\text{T}}$ of all objects, not including the candidate, within a cone of $\Delta R$ around the candidate and dividing by the candidate $p_{\text{T}}$. That is,

[TABLE]

where the sum runs over all the objects $i$ around the candidate within the $\Delta R$ cone.

The objects used in the analysis are defined below.

Photons, $\gamma$, are reconstructed by considering energy deposits in the electromagnetic calorimeter (ECAL) and no tracks in the inner detector. Objects successfully reconstructed as photons are required to have a $p_{\text{T}}>30\,\text{Ge\kern-0.6458ptV}$ and $|\eta|<2.37$. Photons in the transition region are not taken into account. Overlap removals are done in the modified Delphes card as described above, where the photon candidate is identified and put in the correct container by passing the photon efficiency cut corresponding to the ATLAS tight quality efficiency cuts Aaboud:2018yqu . Isolation of the photon is done after the simulation and it is considered isolated when the isolation variable $I<0.008$, where $I$ is defined as described above.

Leptons, $\ell$, are in the following understood to mean electrons or muons only, and not $\tau$-leptons. Electrons are reconstructed by looking at both energy deposit in the ECAL and having a track in the inner tracking system. For the following, simulation in Delphes reconstruction of the electron is done by combining the reconstruction efficiency of the two subsystems and parametrise it as a function of energy and pseudorapidity. Muons pass the calorimeters and are reconstructed by combining the information from the inner tracker and the muon spectrometer. In Delphes, the user specifies the efficiency of the muons such that a muon is only reconstructed with a certain probability delphes . Leptons are required to pass an isolation criterion for which $I<0.12$ within the cone $\Delta R<0.2$ for electrons and $\Delta R<0.3$ for muons. Furthermore, leptons are required to have $p_{\text{T}}>25\,\text{Ge\kern-0.6458ptV}$ and be in the region of $|\eta|<2.47$, excluding the transition region in the case of electrons. Further overlap removals of leptons are done in Delphes where the lepton candidate is identified and put into the correct container by passing the given lepton efficiency. For electrons, the efficiencies correspond to the ATLAS tight quality efficiency cut ATLAS-CONF-2016-024 . For muons, the default Delphes values are used.

$Z$** bosons**, $Z$, are identified as two leptons with same flavour and opposite signs, whose invariant mass fall within the window $|M_{\ell^{+}\ell^{-}}-m_{Z}|<10\,\text{Ge\kern-0.6458ptV}$ where $M_{\ell^{+}\ell^{-}}$ is the invariant mass of the reconstructed leptons.

Jets, $j$, are reconstructed by using the FastJet fastjet package together with Delphes. Here the anti-$k_{t}$ algorithm Cacciari:2008gp with a $R$ parameter of $R=0.4$ is in use for jet reconstruction. Jets are required to pass $p_{\text{T}}>25\,\text{Ge\kern-0.6458ptV}$ and $|\eta|<2.47$, excluding the transition region.

$B$-jets, $j_{b}$, are jets which originate from the hadronisation of a $b$-quark. In Delphes this means a jet which contains a truth $b$-quark. The efficiency and misidentification rate is parametrised in Delphes based on estimates from ATLAS delphes ; ATL-PHYS-PUB-2015-022 .

Missing transverse energy, $E_{\text{T}}^{\text{miss}}$, is computed in Delphes by taking the negative scalar sum of the transverse component of the momenta of all calorimeter towers (i.e., energy deposits in the calorimeter), $\vec{E}_{\text{T}}^{\text{miss}}=-\sum_{i}\vec{p_{\text{T}}}(i)$ delphes .

The scalar transverse energy, $H_{\text{T}}$, is computed by taking the scalar sum of the $p_{\text{T}}$ of all reconstructed basic objects used in the analysis, in this case: jets, muons, electrons and photons. All these objects which enter the $H_{\text{T}}$ definition are required to pass the stated analysis $p_{\text{T}}$ and $\eta$ cuts.

Appendix D Signal efficiencies

In this appendix we present the signal efficiencies for each channel and mass point considered in the analysis, except those already shown in figure 9. Figures 18, 19, 20 and 21 show, respectively, the efficiencies for the $\gamma\,\gamma$ SR and for channels where at least one of the two $S$ decays into $\gamma\,\gamma$, $\gamma\,\gamma$ SR and at least one $S$ decaying to $Z\,\gamma$, $Z\,\gamma$ SR and at least one $S$ decaying to $\gamma\gamma$ and $Z\,\gamma$ SR and at least one $S$ decaying to $Z\,\gamma$.

Acknowledgements.

This work would not have been possible without financing from the Knut and Alice Wallenberg foundation under the grant KAW 2017.0100, which supported EBK, DBF, VE, RE, GF, TMat and LP as a part of the SHIFT project. RB’s work is supported by the Moroccan Ministry of Higher Education and Scientific Research MESRSFC and CNRST: Project PPR/2015/6. RB, RE and SM are supported in part by the H2020-MSCA-RISE-2014 grant agreement No. 645722 (NonMinimalHiggs). TMan is grateful for support from The Royal Society of Arts and Sciences of Uppsala. Y-BL is supported by the Foundation of Henan Institute of Science and Technology (Grant no. 2016ZD01) and the China Scholarship Council (201708410324). SM is supported in part by the NExT Institute and the STFC Consolidated Grant ST/L000296/1. DBF would like to thank the ITP of the Göttingen University and GWDG for providing computing resources. MI wishes to thank the Swedish National Infrastructure for Computing (project SNIC 2019/3-68) for computing resources. LP and Y-BL acknowledge the use of the IRIDIS HPC facility at the University of Southampton.