Charged Higgs Signals in $t\,\overline{t}\,H$ Searches

TL;DR

This paper explores how light charged and neutral Higgs bosons from extended Higgs sectors can be detected via rare top decays, potentially contaminating standard $tar{t}H$ searches and explaining existing excesses.

Contribution

It demonstrates that current $tar{t}H$ analyses can already probe light extended Higgs sectors and proposes methods to distinguish these signals from Standard Model processes.

Findings

Existing $tar{t}H$ searches constrain light extended Higgs sectors.

Rare top decays can mimic $tar{t}H$ signals, affecting search results.

Possible explanation for observed excesses in $tar{t}H$ analyses.

Abstract

New scalars from an extended Higgs sector could have weak scale masses and still have escaped detection. In a Type I Two Higgs Doublet Model, for instance, even the charged Higgs can be lighter than the top quark. Because electroweak production of these scalars is modest, the greatest opportunity for their detection might come from rare top decays. For mass hierarchies of the type $m_t>m_{H^\pm}>m_{A^0,\,H^0}$, the natural signal can arise from top quark pair production, followed by the decay chain $t \rightarrow b\,H^+$, $H^+ \rightarrow W^{+(*)} \phi^0$, $\phi^0\rightarrow b\overline{b},\,\tau^+\tau^-$, where $\phi^0=A^0,\,H^0$. These final states largely overlap with those of the Standard Model $t\,\bar{t}\,H^0_{_\text{SM}}$ process, and therefore can potentially contaminate $t\,\bar{t}\,H^0_{_\text{SM}}$ searches. We demonstrate that existing $t\,\bar{t}\,H^0_{_\text{SM}}$ analyses can already probe light extended Higgs sectors, and we derive new constraints from their results. Furthermore, we note that existing excesses in $t\,\bar{t}\,H^0_{_\text{SM}}$ searches can be naturally explained by the contamination of rare top decays to new light Higgses. We discuss how to distinguish this signal from the Standard Model process.

Tables

Table 1. Table 1: Expected signal yield from t ​ t ¯ ​ ( H SM → τ ​ τ ) 𝑡 ¯ 𝑡 → subscript 𝐻 SM 𝜏 𝜏 t\bar{t}({H_{{}_{\text{SM}}}}\rightarrow\tau\tau) in the signal regions of ATL ( 2016d ) , obtained using Delphes and PGS for simulation of the detector response, as well as the expected yield provided by ATLAS.

	$2\ell 0\tau$	$2\ell 1\tau$	$3\ell$	$4\ell$
$N_{\text{PGS}}$	1.2	1.4	1.7	0.20
$N_{\text{Delphes}}$	1.0	1.1	2.1	0.55
$N_{\text{ATLAS}}$	1.7	0.73	1.2	0.11

Table 2. Table 2: Expected signal yield from t ​ t ¯ ​ ( H SM → W ​ W ∗ ) 𝑡 ¯ 𝑡 → subscript 𝐻 SM 𝑊 superscript 𝑊 t\bar{t}({H_{{}_{\text{SM}}}}\rightarrow WW^{*}) in the signal regions of ATL ( 2016d ) , obtained using Delphes and PGS for simulation of the detector response, as well as the expected yield provided by ATLAS.

	$2\ell 0\tau$	$2\ell 1\tau$	$3\ell$	$4\ell$
$N_{\text{PGS}}$	5.8	1.1	4.8	0.42
$N_{\text{Delphes}}$	6.7	0.80	8.4	1.3
$N_{\text{ATLAS}}$	7.5	0.66	4.6	0.42

Table 3. Table 3: Expected signal yield from t ​ t ¯ ​ ( H SM → Z ​ Z ∗ ) 𝑡 ¯ 𝑡 → subscript 𝐻 SM 𝑍 superscript 𝑍 t\bar{t}({H_{{}_{\text{SM}}}}\rightarrow ZZ^{*}) in the signal regions of ATL ( 2016d ) , obtained using Delphes and PGS for simulation of the detector response, as well as the expected yield provided by ATLAS.

	$2\ell 0\tau$	$2\ell 1\tau$	$3\ell$	$4\ell$
$N_{\text{PGS}}$	0.25	0.06	0.17	0.021
$N_{\text{Delphes}}$	0.37	0.069	0.59	0.16
$N_{\text{ATLAS}}$	0.29	0.03	0.25	0.05

Equations

$$pp\to t\bar{t}$$

$${\varphi }^0\to b\bar{b},{\tau }^{+}{\tau }^{-},$$

$${\mathcal{L}}_{\text{yukawa}}=H_{1}Q{Y}_u{U}^c+H_1^{†}Q{Y}_d{D}^c+H_1^{†}L{Y}_{\ell }{E}^c+\text{h.c.}$$

$$V_{\text{scalar}}$$

$$\displaystyle\left(\hskip-5.0pt\begin{array}[]{r}H_{1}^{0}\@arraycr H_{2}^{0}\end{array}\hskip-5.0pt\right)~{}=~{}\left(\hskip-5.0pt\begin{array}[]{r}v_{1}\@arraycr v_{2}\end{array}\hskip-5.0pt\right)~{}+~{}\frac{1}{\sqrt{2}}R_{\alpha}\left(\hskip-5.0pt\begin{array}[]{r}H^{0}_{\text{light}}~{}\@arraycr H^{0}_{\text{heavy}}\end{array}\hskip-5.0pt\right)~{}+~{}\frac{i}{\sqrt{2}}R_{\beta}\left(\hskip-5.0pt\begin{array}[]{r}G^{0}\@arraycr A^{0}\end{array}\hskip-5.0pt\right)\,,$$

$$\displaystyle\left(\hskip-5.0pt\begin{array}[]{r}H_{1}^{\pm}\@arraycr H_{2}^{\pm}\end{array}\hskip-5.0pt\right)~{}=~{}R_{\beta}\left(\hskip-5.0pt\begin{array}[]{r}G^{\pm}\@arraycr H^{\pm}\end{array}\hskip-5.0pt\right)\,,$$

$$R_{\alpha }=\left(\begin{array}{cc}\cos \alpha & \sin \alpha \\ -\sin \alpha & \cos \alpha \end{array}\right),R_{\beta }=\left(\begin{array}{cc}-\sin \beta & \cos \beta \\ \cos \beta & \sin \beta \end{array}\right).$$

$$\left(\hskip-5.0pt\begin{array}[]{r}H_{1}^{0}\@arraycr H_{2}^{0}\end{array}\hskip-5.0pt\right)~{}=~{}\left(\hskip-5.0pt\begin{array}[]{r}v_{1}\@arraycr v_{2}\end{array}\hskip-5.0pt\right)~{}+~{}\frac{1}{\sqrt{2}}R_{\beta+\delta}\left(\hskip-5.0pt\begin{array}[]{r}H_{{}_{\text{SM}}}^{0}\@arraycr{H^{0}{}}\end{array}\hskip-5.0pt\right)~{}+~{}\frac{i}{\sqrt{2}}R_{\beta}\left(\hskip-5.0pt\begin{array}[]{r}G^{0}\@arraycr{A^{0}}\end{array}\hskip-5.0pt\right)\,,$$

$$R_{\beta -\delta }=\left(\begin{array}{cc}\sin (\beta -\delta )& -\cos (\beta -\delta )\\ \cos (\beta -\delta )& \sin (\beta -\delta )\end{array}\right).$$

$${\mathcal{L}}_{\text{yukawa}}=\sum _f$$

$$+$$

$${\xi }_{{}_{H_{\text{SM}}}}=\cos \delta -\frac{\sin \delta }{\tan \beta },{\xi }_{{}_H}=\sin \delta +\frac{\cos \delta }{\tan \beta },{\xi }_{{}_A}^u=-{\xi }_{{}_A}^{d,e}=\frac{1}{\tan \beta }.$$

$${\mathcal{L}}_{\varphi VV}={\zeta }_{\varphi }\frac{2m_W^2}{v}{\varphi }^0{W}^{+}{W}^{-}+{\zeta }_{\varphi }\frac{m_Z^2}{v}{\varphi }^{0}ZZ,$$

$${\zeta }_{{}_{H_{\text{SM}}}}=\cos \delta ,{\zeta }_H=\sin \delta ,{\zeta }_A=0.$$

$$M_{{\varphi }^0=A^0\text{or}{H}^0}<M_{H_{{}_{\text{SM}}}^0},M_{H^{\pm }}<M_t$$

$$\frac{\text{Br}(H^{\pm}\rightarrow W^{\pm(*)}H^{0})}{\text{Br}(H^{\pm}\rightarrow W^{\pm(*)}A^{0})}\,\Bigg{|}_{m_{A^{0}}=m_{H^{0}}}=~{}~{}~{}1-(\sin\delta)^{2}.$$

$$pp\to \bar{t}t\to \bar{t}b(H^{+}\to W^{+(\ast )}(A^0\to {\tau }^{+}{\tau }^{-})),$$

$$pp\to \bar{t}b(H^{+}\to W^{+(\ast )}(A^0\to {\tau }^{+}{\tau }^{-})).$$

$$S=S_{\text{res}}+S_{\text{nonres}},$$

$$S_{\text{res}}={ϵ}_{\text{res}}\times \left({\sigma }_{t\bar{t}}\times \mathcal{L}\right)\times \left(2\text{Br}(t\to b{H}^{+})\times \text{Br}(H^{+}\to W^{+(\ast )}{A}^0)\times \text{Br}(A^0\to {\tau }^{+}{\tau }^{-})\right),$$

$$S_{\text{nonres}}={ϵ}_{\text{nonres}}\times \left(\frac{{\sigma }_{\text{nonres}}}{\tan {\beta }^2}\times \mathcal{L}\right)\times \left(\text{Br}(H^{+}\to W^{+(\ast )}{A}^0)\times \text{Br}(A^0\to {\tau }^{+}{\tau }^{-})\right).$$

$$\sigma_{\text{nonres}}(m_{H^{\pm}})\,\Big{|}_{\text{13 TeV}}=\left(22.53-0.106\;\frac{m_{H^{\pm}}}{\text{GeV}}\right)\;\text{pb}$$

$$\text{Br}(t\to b{H}^{+})=\frac{R_{H^{+}}}{1+R_{H^{+}}},$$

$$R_{H^{+}}$$

$$p_{H^{+}}=\frac{m_t}{2}\sqrt{{\left(1-\frac{m_{H^{+}}^2}{m_t^2}\right)}^2-2\frac{m_b^2}{m_t^2}\left(1+\frac{m_{H^{+}}^2}{m_t^2}\right)+\frac{m_b^4}{m_t^4}},$$

$$\Gamma (H^{+}\to f{\bar{f}}^{\prime })=\frac{G_F}{4\sqrt{2}\pi }\frac{m_{H^{+}}}{\tan {\beta }^2}{N}_c|U_{f{\bar{f}}^{\prime }}{|}^2\left(m_f^2+m_{{\bar{f}}^{\prime }}^2\right),$$

$$\Gamma (H^{+}\to W^{+}b\bar{b})=$$

$$\Gamma (H^{+}\to W^{+}{\varphi }^0)=(1-{\zeta }_{\varphi }^2)\frac{G_F}{8\sqrt{2}\pi }\frac{m_W^4}{m_{H^{+}}}\sqrt{\lambda (m_{{\varphi }^0}^2,m_W^2,m_{H^{+}}^2)}\lambda (m_{{\varphi }^0}^2,m_{H^{+}}^2,m_W^2),$$

$$\lambda (x,y,z)={\left(1-\frac{x}{z}-\frac{y}{z}\right)}^2-4\frac{xy}{z^2}.$$

$$\Gamma (H^{+}\to W^{+\ast }{\varphi }^0)=(1-{\zeta }_{\varphi }^2)m_{H^{+}}\frac{9G_F^2{m}_W^4}{8{\pi }^3}G\left(\frac{m_{{\varphi }^0}^2}{m_{H^{+}}^2},\frac{m_W^2}{m_{H^{+}}^2}\right),$$

$$G(x,y)$$

Full text

1. I Introduction

2. II Model and Signals

3. II.1 Charged Higgs Decays

4. III Constraints on a Light Higgs Sector

5. III.1 Indirect Bounds

6. III.2 Collider bounds

7. IV Signals in $t\bar{t}{H_{{}_{\text{SM}}}}$ Searches

8. V Recast of the ATLAS 13 TeV Search in Multileptons

9. VI Discussion

10. A Recasting the ATLAS search for $t\bar{t}{H_{{}_{\text{SM}}}}$ in multileptons

11. A.1 Overview

12. A.2 The recasting procedure

13. A.3 Signal generation

14. B Signal yields and statistical procedure

15. B.1 Signal yields

16. B.2 Branching ratios

17. B.3 Statistical treatment of fits and exclusions

I Introduction

So far, LHC searches have not provided conclusive signs of new particles, nor significant deviations from Standard Model predictions. Generic limits on new colored particles are particularly severe, with squarks and gluinos from Supersymmetry already tightly constrained if lighter than 1.3 TeV and 1.9 TeV, respectively ATL (2016a).

Exclusion limits on direct production of new electroweak particles, in contrast, have not been as dramatic. Mass limits on charginos produced via electroweak interactions, for instance, do not extend beyond 400-500 GeV CMS (2016a). The reason for this is straightforward: at a $p$-$p$ machine such as the LHC, electroweak cross sections are simply much smaller than strong cross sections. Because of that, the best means to search for new electroweak states is often in cascade decays of copiously produced colored particles. This approach however relies on the existence of heavier colored particles within the reach of the LHC. As this possibility becomes more and more remote, the only realization of this scenario that can be concretely studied are rare top quark decays, which unfortunately cannot probe new particles heavier than about $170$ GeV.

The range below the top quark mass is nonetheless a well motivated region to search for states beyond the Standard Model (BSM). One possibility of great interest would be rare top decays to charged Higgs bosons, $t\rightarrow b\,{H^{+}}$. With the $t\overline{t}$ cross section at the 13 TeV LHC being over 800 pb, even small branching ratios $\text{Br}(t\rightarrow b\,{H^{+}})\sim\mathcal{O}(10^{-3})$ would yield a $H^{\pm}$ production rate at the $\mathcal{O}$(pb) level or higher. Since direct electroweak production of these states ranges from $\mathcal{O}(30-200)~{}{\rm fb}$ at 13 TeV, the rate from top quark decays could easily dominate.

This scenario has not gone without scrutiny. Indeed, there is a variety of independent constraints on new charged scalars lighter than the top quark. In Type II  Two Higgs Doublet Models (2HDM) in particular, flavor changing observables, most notably $b\rightarrow s\gamma$, exclude m_{H^{\pm}}\begin{array}[]{c}\,\sim\vspace{-26pt}\\ <\end{array}580{~{}\rm GeV} Enomoto and Watanabe (2016); Misiak and Steinhauser (2017), absent cancellations. Flavor bounds are highly model dependent, however. To contrast, in Type I  2HDMs, constraints from $b\rightarrow s\gamma$ are mild at best, requiring only that $\tan\beta\begin{array}[]{c}\sim\vspace{-26pt}\

\end{array}(1.5-2)$for$m_{H^{+}}<m_{t}$. Thus, at least for Type I 2HDM scenarios, there is a compelling case to consider direct searches for new Higgs states at colliders, in particular in rare top decays.

A critical point to consider here is that, in a Type I 2HDM, one of the Higgs doublets is fermiophobic. This can drastically alter the phenomenology of the charged Higgs relative to a Type II scenario. In particular, if a lighter neutral scalar $\phi^{0}\,(=A^{0},\,H^{0})$ exists, the charged Higgs will dominantly decay as ${H^{+}}\rightarrow\phi^{0}\,W^{+(*)}$, and the familiar fermionic decays (e.g., ${H^{+}}\rightarrow\tau^{+}\nu,\,c\,\bar{s},\,t^{*}\,\overline{b}$) will be suppressed. In this region of parameter space, our process of interest will be:

[TABLE]

with $\phi^{0}$ dominantly decaying as

[TABLE]

if m_{\phi^{0}}\begin{array}[]{c}\,\sim\vspace{-26pt}\\ <\end{array}110~{}\text{GeV}. The resulting final state, with rates of $\mathcal{O}$(pb) or higher, can lead to a large signal contamination in searches for the SM Higgs boson produced in association with top quark pairs, $t\bar{t}{H_{{}_{\text{SM}}}}$, whose SM cross section is about $0.5~{}\text{pb}$. Indeed, existing and future measurements of the $t\bar{t}{H_{{}_{\text{SM}}}}$ cross section can be used to constrain the Type I 2HDM signatures in (1.1), (1.2). A more exciting prospect would be to explain recent excesses in existing $t\bar{t}{H_{{}_{\text{SM}}}}$ searches as due to contamination from rare top decays to charged Higgses. While the significance of current excesses is mild, upcoming results with more data will lead to a clearer picture of the excess pattern, would it persist.

The layout of this paper is as follows: in Sec. II we briefly review Type I 2HDMs and describe the charged Higgs phenomenology in the light mass region. Direct and indirect bounds on the relevant region of parameter space are reviewed in Sec. III. In Secs. IV, V, we discuss how $t\bar{t}{H_{{}_{\text{SM}}}}$ searches can be used to constrain charged Higgs production, and describe the degree to which the claimed excess in various channels can be explained by this model. Finally, in Sec.VI we discuss the implications of these results and comment on future searches that might help better constrain this light region of Type I 2HDMs.

II Model and Signals

In a type I 2HDM, one of the Higgs doublets, $H_{2}$, is fermiophobic, and all fermion masses stem from Yukawa couplings to $H_{1}$:

[TABLE]

The scalar potential can be generically parameterized as Gunion et al. (2000):

[TABLE]

where both doublets, $H_{1}$ and $H_{2}$, have hypercharge $Y=1/2$, and for simplicity we assume that CP is conserved and all parameters in (II) are real.

Conventionally, the mass eigenstates of this theory are parameterized by two angles; $\alpha$, the mixing angle between the CP-even neutral states, and $\beta$, defined as $\text{tan}\beta\equiv v_{1}/v_{2}$:

[TABLE]

[TABLE]

where

[TABLE]

Usually, the “SM-like” Higgs (corresponding to the state discovered at the LHC with mass $m_{h}=125$ GeV) is the lighter CP-even neutral scalar, $H^{0}_{\text{light}}$. That does not need to be the case though, and in principle the SM-like Higgs could be the heavier CP-even scalar, $H^{0}_{\text{heavy}}$. Since we are interested in both regimes, we will adopt the following, more generic parameterization Alves et al. (2012)

[TABLE]

where

[TABLE]

Here, $\delta$ is a parameter that describes the deviation from the alignment limit. If the SM-like Higgs corresponds to the lightest CP-even scalar, $\delta$ is defined by $\delta\equiv\beta-\alpha-\pi/2$. Conversely, if the SM-like Higgs corresponds to the heaviest CP-even scalar, then $\delta\equiv\beta-\alpha$. The advantage of this parameterization is that $\delta$ quantifies the deviation from a SM-like Higgs, and there is no discontinuity in our description of fields as the mass hierarchy changes. That is, $H_{{}_{\text{SM}}}^{0}$ is always the SM-like state, and $H^{0}$ is always the state with suppressed couplings to fermions.

In terms of the mass eigenstates, the Yukawa couplings can be re-written as:

[TABLE]

where $v=246$ GeV, $U_{ff^{\prime}}$ is a CKM matrix element if $f,f^{\prime\,c}$ are quarks, and $U_{ff^{\prime}}=1$ if $f,f^{\prime\,c}$ are leptons. Moreover,

[TABLE]

Likewise, the EW symmetry breaking couplings of scalars to vector bosons are given by:

[TABLE]

where $\phi^{0}$ generically denotes $H^{0}_{{}_{\text{SM}}}$, $H^{0}$, $A^{0}$, and,

[TABLE]

In Type I 2HDMs, unlike Type II, the couplings of $A^{0}$ and $H^{\pm}$ to fermions are suppressed by $\rm tan\beta$, as shown in (II) and (2.15), and so are the $H^{0}$ yukawa couplings in the alignment limit $\delta\rightarrow 0$.

As we are especially interested in $H^{\pm}$ production from top decays, we show in Fig. 1 the branching ratio $\text{Br}(t\rightarrow b\,{H^{+}})$ as a function of $m_{H^{\pm}}$ and $\rm tan\beta$. Note that even at large $\rm tan\beta\sim\mathcal{O}(10)$, branching ratios of $\mathcal{O}(10^{-3})$ are possible, while at low \rm tan\beta\begin{array}[]{c}\,\sim\vspace{-26pt}\\ <\end{array}3, the top branching ratio to $H^{\pm}$ can reach the few percent level, $\text{Br}(t\rightarrow b\,{H^{+}})\sim\mathcal{O}(3\%)\;-$ roughly the limit where tension might arise with measurements of the top pair cross section $\sigma_{t\bar{t}}$ Czakon and Mitov (2014); Peters (2015); Andrea (2013); Schilling (2012); ATL (2012a, b); CMS (2013, 2014a); LHC Top Physics Working Group .

II.1 Charged Higgs Decays

The decay patterns of the charged Higgs can vary dramatically across the parameter space of 2HDMs, significantly impacting the experimental strategies to search for this state.

Due to the large popularity of Type II 2HDMs, the most thoroughly explored $H^{\pm}$ decays, both in phenomenological studies as well as in experimental efforts, have been the fermionic modes. This is due to the $\rm tan\beta$ enhancement of the $H^{\pm}$ couplings to down type quarks and leptons, causing the $\tau\nu$, $cs$ and $b\,t^{*}$ modes to dominate the $H^{\pm}$ branching ratios. ATLAS Aad et al. (2015); Aaboud et al. (2016) and CMS CMS (2014b, 2016b) have extensively searched for signatures of $t\rightarrow b\,(H^{+}\rightarrow\tau^{+}\nu)$, and placed upper bounds on the overall top branching ratio to this final state at the sub-percent level. Analogous searches Khachatryan et al. (2015); CMS (2016c) for the $H^{+}\rightarrow c\bar{s},\,c\bar{b}$ channels have also set percent-level bounds on the corresponding branching ratios.

The same constraints are applicable in Type I 2HDMs if the only kinematically open channels for $H^{\pm}$ decays are light fermions, which is the case if m_{H^{\pm}}\begin{array}[]{c}\,\sim\vspace{-26pt}\\ <\end{array}m_{H^{0}},m_{A^{0}}. However, if the spectrum contains a lighter neutral scalar with a large $H_{2}$ component, such as $A^{0}$ or $H^{0}$, the mode ${H^{+}}\rightarrow W^{\pm(*)}\,A^{0}/H^{0}$ would naturally dominate the $H^{\pm}$ branching ratio, even with the 3-body phase space suppression of an off-shell $W^{*}$. This is due to the unsuppressed gauge couplings that mediate this decay mode, and the large suppression of the competing fermionic modes, stemming from the smallness of the Yukawa couplings and from the $1/\rm tan\beta$ dependence of the $H^{\pm}$ coupling to fermions. This effect is illustrated in Fig. 2, where we have set $m_{A^{0}}=m_{H^{0}}=100$ GeV, $\rm tan\beta=3$, and $\delta=0$. The suppression of fermionic modes is even more pronounced for larger values of $\rm tan\beta$, or lower masses of $A^{0}$ or $H^{0}$. This qualitatively different phenomenology of charged Higgs decays has been previously noted in phenomenological studies Djouadi et al. (1996); Akeroyd (1999); Kling et al. (2015); Coleppa et al. (2014a, b); Arhrib et al. (2016); Bechtle et al. (2017), but to the best of our knowledge, no dedicated experimental analysis has explicitly searched for these signatures. A critical question is then: could such a particle have contaminated studies of other SM or BSM processes? And if so, what constraints could existing searches place on these particular charged Higgs signals?

III Constraints on a Light Higgs Sector

While new scalars with sizeable couplings to SM fermions or gauge bosons are subject to constraints from LEP, Tevatron, and LHC data, additional Higgs bosons with suppressed yukawa couplings are more elusive to existing searches. In this section, we summarize the constraints on the charged and neutral Higgs bosons of Type I 2HDMs, with a focus on the light mass region.

III.1 Indirect Bounds

As previously mentioned, indirect bounds on light Type I 2HDMs are mild. Besides $b\rightarrow s\gamma$ already discussed in Sec. I, other competing constraints from flavor observables are $B_{s}\rightarrow\mu^{+}\mu^{-}$ and $\Delta M_{B_{s,d}}$, which are only marginally stronger, requiring that $\rm tan\beta\begin{array}[]{c}\sim\vspace{-26pt}\

\end{array}1.8-2.2$in the mass range$m_{H^{\pm}}<m_{t}$ Enomoto and Watanabe (2016).

Another source of indirect constraints comes from contributions to the electroweak oblique parameters, particularly $T$, induced by the mass splittings between $H^{\pm}$, $A^{0}$ and $H^{0}$. For the light spectra considered here, however, we have checked that $\Delta T$ constraints are easily evaded.

III.2 Collider bounds

LEP placed a robust lower bound on $m_{H^{\pm}}\begin{array}[]{c}\sim\vspace{-26pt}\

\end{array}78.6$GeV by searching for the decay modes$H^{+}\rightarrow c\bar{s},,\tau^{+}\nu, assuming the absence of any non-fermionic decays Sea ([2001](#bib.bib29)). The DELPHI and OPAL collaborations also considered the bosonic decay H^{\pm}\rightarrow W^{\pm*}A^{0} Abdallah *et al.* ([2004](#bib.bib30)); Abbiendi *et al.* ([2012](#bib.bib31)). In Type I 2HDM scenarios, the \rm tan\beta$-independent limits obtained were$m_{H^{\pm}}\begin{array}[]{c}\sim\vspace{-26pt}\ \end{array}77.6$GeV (DELPHI) and$m_{H^{\pm}}\begin{array}[]{c}\sim\vspace{-26pt}\ \end{array}65$GeV (OPAL), provided that$m_{A^{0}}\begin{array}[]{c}\sim\vspace{-26pt}\ \end{array}12$ GeV.

The OPAL collaboration has searched for the associated production $e^{+}e^{-}\rightarrow A^{0}H^{0}$, with $A^{0},\,H^{0}\rightarrow q\bar{q},\,gg$, and $\tau^{+}\tau^{-}$ Abbiendi et al. (2005, 2004). While the resulting mass limits vary across the parameter space, they essentially turn off for either $A^{0}$ or $H^{0}$ heavier than $\sim 80{~{}\rm GeV}$.

LEP, Tevatron and LHC searches for the SM Higgs are also potentially sensitive to the neutral scalars, $A^{0}$ and $H^{0}$. If $A^{0},\,H^{0}$ are lighter than $\sim 110{~{}\rm GeV}$, however, these states decay dominantly to $b\bar{b}$ final states (with $\tau^{+}\tau^{-}$ as the subleading mode), and are challenging to probe at the Tevatron and LHC due to the large backgrounds and suppressed cross sections relative to the SM Higgs. While SM Higgs searches at LEP cannot constrain $A^{0}$ due to the absent $A^{0}Z^{0}Z^{0}$ coupling (see Eqs. (2.16,2.17)), they are sensitive to $e^{+}e^{-}\rightarrow H^{0}Z^{0}$ production, and constrain \zeta_{H}^{2}\begin{array}[]{c}\,\sim\vspace{-26pt}\\ <\end{array}0.01-0.3 in the range $m_{H^{0}}\simeq(15-115){~{}\rm GeV}$ Barate et al. (2003).

Furthermore, LHC measurements of the SM Higgs properties provide non-trivial constraints on the parameters of the scalar potential (II). If $\phi^{0}$ ($=A^{0}$ or $H^{0}$) is ligher than $m_{H_{\text{SM}}}/2$, the decay channel ${H_{{}_{\text{SM}}}}\rightarrow\phi^{0}\phi^{0}$ can easily dominate the SM Higgs width for generic values of the quartic couplings in (II). In order to avoid conflict with observations, in the mass range $m_{\phi^{0}}\lesssim 62$ GeV the tri-scalar coupling $\lambda_{\phi\phi{H_{{}_{\text{SM}}}}}$ must be suppressed, $\lambda_{\phi\phi{H_{{}_{\text{SM}}}}}\lesssim(2-6)$ GeV. While this condition is not generically satisfied, it is a parameter that can be adjusted independently of the physical masses and mixing angles $\delta$ and $\beta$. Since the charged Higgs phenomenology we will consider is not directly affected by $\lambda_{\phi\phi{H_{{}_{\text{SM}}}}}$, we will include the region $m_{\phi^{0}}\lesssim 62$ GeV in our study, with the implicit assumption that $\Gamma({H_{{}_{\text{SM}}}}\rightarrow\phi^{0}\phi^{0})$ is not in conflict with observations.

Another parameter that is directly constrained by SM Higgs measurements is $\delta$ – current data pushes the model towards the alignment limit where $\delta$ is small and the properties of the 125 GeV Higgs are “SM-like”. For a more thorough discussion on that, see Alves et al. (2012); Bernon et al. (2015, 2016); Haber (2017).

Previously mentioned upper bounds on $\text{Br}(t\rightarrow b\,(H^{+}\rightarrow\tau^{+}\nu))$ from ATLAS and CMS are sensitive enough to be relevant even if the decay mode $H^{+}\rightarrow\tau^{+}\nu$ is subdominant, as is the case of the Type I 2HDMs we are considering. We recast the 8 TeV CMS constraints on the branching ratio $\text{Br}(t\rightarrow b\,H^{+})\times\text{Br}(H^{+}\rightarrow\tau^{+}\nu)$ CMS (2014b) as lower bounds on the value of $\rm tan\beta$, displayed in Fig. 3 as a function of $m_{H^{\pm}}$ and $m_{A^{0}}$, assuming for simplicity (and without loss of generality) that $m_{H^{0}}>m_{H^{\pm}}$.

To summarize, the light mass region of Type I 2HDMs is still experimentally viable in a vast swath of parameter space. In what follows, we investigate how this region can be constrained by existing LHC searches for the Standard Model $t\,\bar{t}\,H_{{}_{\text{SM}}}$ process.

IV Signals in $t\bar{t}{H_{{}_{\text{SM}}}}$ Searches

Although the parameter space of Type I 2HDMs with mass hierarchy:

[TABLE]

is still experimentally viable, its phenomenology remains relatively unexplored in comparison to that of heavier 2HDM spectra. A stricking signal of models with (4.1) are rare top decays that can contaminate $t\bar{t}{H_{{}_{\text{SM}}}}$ searches, particularly those targeting leptonic or $b\bar{b}$ decays of the SM Higgs, as illustrated in Fig. 4. Given the very large top pair cross section, and the fact that $\text{Br}(t\rightarrow bH^{+})$ can be as high as a few percent, this contamination can lead to observable excesses in $t\bar{t}{H_{{}_{\text{SM}}}}$ measurements relative to the SM expectation. The excess pattern, however, would appear inconsistent across different channels if interpreted as an enhanced $t\bar{t}{H_{{}_{\text{SM}}}}$ signal strength. Generically, no excess should appear on $\gamma\gamma$ channels, since searches typically require $m_{\gamma\gamma}\approx 125$ GeV (within resolution) to specifically target the SM Higgs. On the other hand, one would expect excesses in channels targeting $b\bar{b}$ and $\tau^{+}\tau^{-}$ final states, albeit with different strengths. Many of these analyses use Multivariate discriminants (MVAs), such as boosted decisions trees, neural networks, etc., which may be tuned to the specific final state kinematics of $t\bar{t}{H_{{}_{\text{SM}}}}$. In those cases, the contamination from rare top decays may be partially filtered out, depending on how well the MVAs can discriminate between the two processes. Normally, details of MVA based studies are not public, and the extraction of limits on contaminant signals is unfeasible.

In fact, SM Higgs searches as early as the Tevatron’s could have been contaminated by rare top decays. The CDF collaboration has searched for $t\,\bar{t}\,(h^{0}\rightarrow b\,\bar{b})$ over the mass range $m_{h^{0}}=(100-150)$ GeV Aaltonen et al. (2012, 2013), and observed an $\mathcal{O}(2\sigma)$ excess above expectations at $m_{h^{0}}\sim(100-105)$ GeV. The reported best fit for the signal strength was $\mu_{t\bar{t}h^{0}}=7.40^{+4.65}_{-3.80}$  for $m_{h^{0}}=100~{}{~{}\rm GeV}$, and $\mu_{t\bar{t}h^{0}}=8.56^{+4.82}_{-4.10}$  for $m_{h^{0}}=105~{}{~{}\rm GeV}$. This would correspond to a rate of roughly $(65\pm 40)$ fb, or, in terms of the inclusive top pair cross section, $(0.003-0.015)\times\sigma_{t\,\bar{t}}$. This analysis was MVA based, and its mass resolution was limited due to the presence of four $b$-quarks in the signal final state, leading to a combinatoric ambiguity in identifying the $b$-jets originating from $h^{0}$ decays, and therefore to a broadening of the expected $m_{b\bar{b}}$ peak. All these factors preclude us from inferring any concrete implications regarding a potential contamination from BSM processes, but the results are nonetheless intriguing, and, if corroborated with further deviations at the LHC, could warrant a re-analysis of the Tevatron’s data.

At the LHC, existing $t\bar{t}{H_{{}_{\text{SM}}}}$ results from ATLAS and CMS do seem to suggest a pattern of excesses inconsistent with the hypothesis of enhanced $t\bar{t}{H_{{}_{\text{SM}}}}$ production, although with small statistical significance. At 8 TeV, the uncertainties in $t\bar{t}{H_{{}_{\text{SM}}}}$ measurements are too large to offer any indication of an excess or lack thereof ***One notable exception is the CMS measurement in the same-sign dilepton channel Khachatryan et al. (2014), which gives the following best fit for the signal strength: $\mu_{ttH}=5.3^{+2.1}_{-1.8}$ ., with a combined best fit of $\mu_{ttH}=2.3^{+0.7}_{-0.6}$ (ATLAS + CMS, all channels) Aad et al. (2016a). Existing 13 TeV data is inconclusive as well, showing no excess in CMS $b\bar{b}$ (MVA) CMS (2016d) and ATLAS $\gamma\gamma$ ATL (2016b), and $\mathcal{O}(1\sigma)$ excesses in CMS multileptons (MVA) CMS (2016e, 2017), CMS $\gamma\gamma$ CMS (2016f), ATLAS $b\bar{b}$ (MVA) ATL (2016c), and ATLAS multileptons ATL (2016d). Of all 13 TeV searches to date, only the latter, ATLAS multileptons, employs a traditional cut-and-count procedure†††The $4\ell$ category in the CMS multilepton analysis CMS (2017) employs a cut-and-count procedure as well, but the final measurement has uncertainties substantially larger than the ones in ATLAS, and therefore, is less sensitive., and therefore is amenable to recasting in terms of our charged Higgs signal. We shall do so in the following section.

We end by commenting on our choice of spectrum when reinterpreting the ATLAS multilepton results. As previously discussed, if (4.1) is realized, the charged Higgs will dominantly decay to $W^{(*)}\phi^{0}$, where $\phi^{0}$ is the lightest neutral scalar. Since LHC measurements of the 125 GeV Higgs push this model into the alignment limit, (\sin\delta)^{2}\begin{array}[]{c}\,\sim\vspace{-26pt}\\ <\end{array}0.1, the signals in Fig. 4 will be essentially independent of whether $\phi^{0}=A^{0}$ or $H^{0}$, since

[TABLE]

Moreover, if lighter than $\sim 110$ GeV, $A^{0}$ and $H^{0}$ will have the same leading branching ratios, namely, $\text{Br}(\phi^{0}\rightarrow b\overline{b})\approx 0.8$ and $\text{Br}(\phi^{0}\rightarrow\tau^{+}\tau^{-})\approx 0.08$. For practical purposes, therefore, we will choose $A^{0}$ as the lightest neutral scalar and decouple $H^{0}$ (i.e., set $m_{H^{0}}>m_{H^{\pm}}$) for the remainder of the paper. None of the results that follow change in any significant way if $A^{0}\rightarrow H^{0}$. The only loss of generality that comes with this assumption is the possibility of two independent decay modes. However, ATL (2016d) does not rely on a mass peak, the efficiencies do not vary dramatically over the range of $m_{A^{0},\,H^{0}}$ considered, and, in the presence of a mass hierarchy between $A^{0}$ and $H^{0}$, the lighter mode dominates the bosonic decays of $H^{\pm}$. Consequently, even this complication should not impact our results significantly.

V Recast of the ATLAS 13 TeV Search in Multileptons

At the time of writing of this paper, ATL (2016d) was the most recent ATLAS search for $t\bar{t}{H_{{}_{\text{SM}}}}$ production in the multilepton channel, corresponding to $13.2~{}\text{fb}^{-1}$ of 13 TeV data. It targeted leptonic decays of the SM Higgs in $WW^{*}$, $\tau^{+}\tau^{-}$, and $ZZ^{*}$, by looking into four exclusive signal regions, namely, same-sign dileptons and one hadronically decaying $\tau$ ($2\ell 1\tau_{\text{had}}$), same-sign dileptons vetoing hadronically decaying $\tau$’s ($2\ell 0\tau_{\text{had}}$), 3 leptons ($3\ell$), and 4 leptons ($4\ell$). Rare top decays in Fig. 4, with $A^{0}\rightarrow\tau^{+}\tau^{-}$, can contaminate all of these signal regions. We performed a Monte Carlo (MC) study to obtain a quantitative estimate of this contamination and the sensitivity of ATL (2016d) to the Type I 2HDM spectra of (4.1). Overall, we were able to validate our MC simulation of ATL (2016d) by reproducing the $t\bar{t}{H_{{}_{\text{SM}}}}$ efficiencies provided by ATLAS to within a factor of 2 (12 independent efficiencies in total). This translates into a factor of $\sim 2$ uncertainty in our results for the signal strength, and a $\sim 50\%$ uncertainty in our results for $\rm tan\beta$ (since our signal scales as $\rm tan\beta^{-2}$). For a detailed description of our MC study, as well as our statistical treatment of fits and exclusions, we refer the reader to Appxs. A, B. Throughout our study, we include the contribution of the SM $t\bar{t}{H_{{}_{\text{SM}}}}$ process, assuming $\mu_{t\bar{t}H}=1$.

We first obtain upper bounds on the contamination from $t\rightarrow b{H^{+}}$, so that the expected number of events in the signal regions of ATL (2016d) are compatible with observations. We interpret these constraints as a lower bound on $\rm tan\beta$ in the $m_{H^{+}},\,m_{A^{0}}$ mass plane, shown in Fig. 5(a). Our signal yield is suppressed in the compressed regions m_{H^{\pm}}-m_{A^{0}}\begin{array}[]{c}\,\sim\vspace{-26pt}\\ <\end{array}30 GeV (where $\text{Br}(H^{\pm}\rightarrow W^{\pm(*)}A^{0})$ is small), and m_{t}-m_{H^{\pm}}\begin{array}[]{c}\,\sim\vspace{-26pt}\\ <\end{array}20 GeV (where $\text{Br}(t\rightarrow b\,H^{+})$ is small). In these regions of suppressed signal yield, the inferred lower bounds on $\rm tan\beta$ are innocuous.

These limits would be stronger but for the presence of excesses in the data. Consequently, it behooves us to see whether we can understand these excesses as arising from rare top decays. In Fig. 5(b) we show the best fit for $\rm tan\beta$ at each point in the parameter space of $m_{H^{+}}$ and $m_{A^{0}}$. Notably, the efficiencies for the process $t\rightarrow b\,{H^{+}}\rightarrow bW^{+}(A^{0}\rightarrow\tau^{+}\tau^{-})$ (before folding in branching ratios) do not vary dramatically across the $m_{H^{+}},\,m_{A^{0}}$ mass plane.

In Fig. 6, we show the preferred regions of parameter space, defined from the goodness of fit to the data in ATL (2016d) . Here, the values of $\rm tan\beta$ are profiled at each mass point to yield the best fit (see Fig. 5(b)). Under this assumption, some regions of parameter space are excluded by other measuments. In particular, the compressed region m_{H^{\pm}}-m_{A^{0}}\begin{array}[]{c}\,\sim\vspace{-26pt}\\ <\end{array}40 GeV is in tension with the CMS bounds on $\text{Br}(t\rightarrow b\,({H^{+}}\rightarrow\tau^{+}\nu))$. Likewise, the compressed region m_{t}-m_{H^{\pm}}\begin{array}[]{c}\,\sim\vspace{-26pt}\\ <\end{array}10 GeV is in tension with $b$ flavor observables discussed in Sec. III.1.

Finally, we select two benchmark points to illustrate the pattern of contamination across the four signal regions of ATL (2016d). The first benchmark point, B1, is at low mass, $m_{H^{\pm}}=130$ GeV, $m_{A^{0}}=40$ GeV. For this benchmark, $H^{\pm}$ can decay to an on-shell $W^{\pm}$, yielding a harder charged lepton. Direct production of $H^{+}H^{-}$ and $H^{\pm}A^{0}$ is several hundreds of femtobarns at 13 TeV, and in principle dedicated searches in final states with 1 or 2 leptons plus 3 or more $b$-jets could be sensitive to this point.

The second benchmark, B2, is at high mass, and close to the excluded compressed regions, $m_{H^{\pm}}=160$ GeV, $m_{A^{0}}=95$ GeV. Because of the small mass splitting between $t$ and $H^{+}$, the $b$-jet from $t\rightarrow b\,H^{+}$ is relatively soft and has a lower probability of passing the $b$-tagging requirements. Overall, B2 predicts that roughly 72% of its signal in the $2\ell 1\tau_{\text{had}}$ region has only one $b$-tagged jet. Indeed, the data in the $2\ell 1\tau_{\text{had}}$ channel seem to indicate that the excess appears exclusively in the 1 $b$-tag category. Since current statistics is very low, this might be due to Poisson fluctuations, and only more data will be able to settle this matter.

A final comment on B2 regards our choice of $m_{A^{0}}$ for this benchmark. This was motivated by an excess at $m_{H^{0}}\approx(90-100)$ GeV in the combination of LEP seaches for the SM Higgs Barate et al. (2003), which could be interpreted in our scenario as a signal of $H^{0}$, if this were the lightest neutral scalar with $\zeta_{H}^{2}\sim\mathcal{O}(10^{-1})$ (see Eqs.(2.16),(2.17)).

In Fig. 7, we show the predicted signal strength from benchmarks B1 and B2, measured in units of the SM $t\bar{t}{H_{{}_{\text{SM}}}}$ signal strength, for each of the four signal regions of ATL (2016d). Both the combined best fit, as well as a selected range of $\rm tan\beta$ are shown. Specifically, this range is chosen so it encompasses the $1\sigma$ range favored by data in the $2\ell 1\tau_{\text{had}}$ channel. We point out that for B2, it is not possible to reach the $1\sigma$ upper range of $2\ell 1\tau_{\text{had}}$ without violating flavor observables. For this reason, we cut off the range of B2 at this exclusion boundary.

We can see that the typical excess pattern seen by ATL (2016d) is well explained by the hypothesis of contamination from rare top decays. At this point, however, the uncertainties are still large enough that even the no signal hypothesis is marginally consistent with observations. Upcoming analyses with more data will either tighten the exclusions of our model, or, optimistically, corroborate the deviations from the SM expectation.

VI Discussion

While the LHC has so far found no compelling evidence for new physics, tremendous possibilities still exist for discovery of new particles, including light electroweakly charged ones.

In Type I two Higgs doublet models, a charged Higgs lighter than the top quark can naturally be produced at significant rates from rare top decays. Remarkably, if there are additional light neutral scalars in the spectrum, the final state for such signals has a substantial overlap with those of SM $t\bar{t}{H_{{}_{\text{SM}}}}$ processes. As a consequence, it is natural to consider signals from light extended Higgs sectors as a contaminant to existing SM searches.

Interestingly, many - but not all - of the existing $t\bar{t}{H_{{}_{\text{SM}}}}$ searches show excesses, both at the Tevatron as well as the 8 and 13 TeV runs of the LHC. It is challenging to simultaneously reconcile these excesses with each other - significant excesses in leptonic channels, and an inconclusive pattern in $\gamma\gamma$ and $b\bar{b}$ channels, for instance. If these excesses persist, they could potentially be explained by the contamination of a new, charged Higgs signal from top decays.

On general grounds, one would expect that the more tuned a given analysis is to the specific final state kinematics of the $t\bar{t}{H_{{}_{\text{SM}}}}$ process, the less sensitive it should be to BSM top decays. That would be the case of LHC searches for $t\bar{t}({H_{{}_{\text{SM}}}}\rightarrow\gamma\gamma)$, as these focus on a narrow $m_{\gamma\gamma}$ window around 125 GeV; or of (post Higgs discovery) MVA analyses in general. On the other hand, more inclusive analyses should be more prone to contamination from signals of extended Higgs sectors. Examples of more inclusive analyses are the Tevatron’s CDF search for $t\bar{t}(h^{0}\rightarrow b\bar{b})$ Aaltonen et al. (2012, 2013), which considers a broader $m_{b\bar{b}}$ window of $100-150$ GeV; and the 13 TeV ATLAS search for leptonic $t\bar{t}{H_{{}_{\text{SM}}}}$ ATL (2016d), which employs a more conventional cut-and-count strategy, and might have non-negligible acceptance to BSM signals in multileptons plus two or more $b$-jets.

We have recast the 13 TeV ATLAS search for $t\bar{t}{H_{{}_{\text{SM}}}}$ in multilepton final states ATL (2016d), and found that its signal regions can be naturally contaminated by a light Type I 2HDM spectrum at low $\rm tan\beta$. Our recast of the results of ATL (2016d) provides new limits on these models. Furthermore, these models can also explain the excesses observed in the data without exceeding null results from other measurements. In principle, this signal could also show up in other searches, such as those targeting ${H_{{}_{\text{SM}}}}\rightarrow b\bar{b}$, depending on the details of the MVA used, and the masses of ${A^{0}}/H^{0}$. Indeed, considering the high branching ratio of ${A^{0}}/H^{0}\rightarrow b\bar{b}$, final states with many $b$-jets may provide the strongest tests going forward.

Should the excesses persist, it is clear that light Type I 2HDM spectra provide an attractive potential explanation. Broadening search windows, especially for ${A^{0}}/H^{0}\rightarrow b\bar{b}$ at lower $m_{b\bar{b}}$, could further constrain this scenario, or, possibly, provide the first evidence at the LHC of physics beyond the Standard Model.

Acknowledgements.

We thank A. Djouadi and M. Spira for answering our questions regarding charged Higgs 3-body decays, and Kyle Cranmer for helpful discussions on statistics. We also thank A. Pierce and M. Graesser for interesting conversations about the ATLAS $ttH$ excess. NW and DA are supported by the NSF under grants PHY-0947827 and PHY-1316753. SEH is supported by the Cluster of Excellence Precision Physics, Fundamental Interactions and Structure of Matter (PRISMA-EXC 1098) and by the Mainz Institute for Theoretical Physics.

Appendix A Recasting the ATLAS search for $t\bar{t}{H_{{}_{\text{SM}}}}$ in multileptons

A.1 Overview

The production of a Higgs boson in association with two top quarks is expected in the SM and has therefore been searched for by ATLAS and CMS ATL (2016c); CMS (2016d); ATL (2016d); CMS (2016e). In our study, we focus on the ATLAS $13$ TeV search in multilepton final states ATL (2016d), since it is based on a traditional cut-and-count analysis that is possible to recast. This search is targeted at events where the Higgs decays to either $WW^{*}$, $ZZ^{*}$, or $\tau^{+}\tau^{-}$. All these channels can lead to signatures with up to four leptons, for which the backgrounds are extremely low at the LHC. The potentially interesting events are grouped into four different signal regions: two with a pair of same-sign light leptons and either zero or one hadronic tau ($2\ell 0\tau$ and $2\ell 1\tau$), one with three light leptons ($3\ell$) and one with four light leptons ($4\ell$). The $13.2~{}\text{fb}^{-1}$ data have shown an excess in both the $2\ell 1\tau$ and the $2\ell 0\tau$ regions, more particularly for events with only one $b$-tagged jet. The peculiar structure of this excess cannot be explained solely by an enhanced $t\bar{t}{H_{{}_{\text{SM}}}}$ cross section and could be a sign for new physics.

Naively, the $(bW)(bW^{*}\tau\overline{\tau})$ final state associated with our Type I 2HDMs is similar to the $t\overline{t}({H_{{}_{\text{SM}}}}\rightarrow\tau\overline{\tau})$ signal that is looked for in this search. The kinematics of the final states is however significantly different. Notably, only one $W$ boson arises from the direct decay of a top quark. The other one is produced through the decay of the charged Higgs to ${H^{0}}/{A^{0}}$ and is therefore much softer or even off-shell. Consequently, when the mass splitting between $H^{+}$ and ${H^{0}}/{A^{0}}$ is small, events with $4$ leptons will have lower efficiencies for trigger and preselection cuts. Similarly, the efficiency of our signal in the $3\ell$ region should be lower than that of $t\overline{t}{H_{{}_{\text{SM}}}}$. The observed rates for the $2\ell 0\tau$ and $2\ell 1\tau$ regions should however remain significant. Broadly speaking, this structure is similar to the excess observed in ATLAS.

A.2 The recasting procedure

In the ATLAS $t\bar{t}{H_{{}_{\text{SM}}}}$ search, events are selected and classified into four exclusive signal regions associated with various cuts on the multiplicity and momenta of leptons, hadronic taus, light jets and $b$-jets. In addition to these cuts, vetoes against $m_{\ell\ell}\simeq m_{Z}$ as well as low mass leptonic Drell-Yan are imposed in events with same-flavor lepton pairs. Depending on the signal region, the light flavor leptons can also be required to verify specific isolation criteria. Although most of the selection cuts are straightforward, the tagging of hadronic taus as well as the isolation requirements on the light leptons require a more careful treatment that we detail in what follows.

In the $2\ell 0\tau$, $2\ell 1\tau$ and $3\ell$ regions, the light leptons have to pass loose and tight isolation cuts. Since these cuts are associated with efficiencies of $99$% and $96$% respectively Aad et al. (2016b), we do not implement them in our analysis. In the $4\ell$ region, the electrons and muons are submitted to so-called “gradient” isolation cuts, with a $p_{T}$-dependent efficiency. These cuts can have a significant impact at low $p_{T}$ and we therefore take them into account when using Delphes deF (2014) for detector simulation.

To estimate the errors associated with our modeling of the detector response, we use two different detector simulators: Delphes 3 and PGS 4 deF (2014); Conway . In Delphes, we set the $b$- and $\tau$-tagging efficiencies/mistag rates to the values used by ATLAS in ATL (2016d) and initially loosen the electron and muon isolation criteria. The rest of the parameters are set to the default values given in the ATLAS $13$ TeV card from Delphes. In order to take into account the possible loss of low $p_{T}$ electrons or muons in the $4\ell$ signal region, we select the light leptons in this region with efficiencies corresponding to the ones of the gradient isolation cuts. When generating events with PGS, we use the default lepton tagging algorithm and do not apply any additional isolation cuts. For hadronic taus, however, since the efficiency of the PGS tagging algorithm is much lower than the one used by ATLAS, we modify the PGS code to identify all jets within $\Delta R=0.2$ of a parton-level tau as hadronic taus. We apply a flat tau-tagging efficiency ex post facto, given by the branching-ratio-weighted average of the one-prong and three-prong efficiencies given in ATL (2016d). Likewise, we modify the PGS $b$-tagging algorithm to better represent the working point used in ATL (2016d). All the other cuts besides the preselection cuts are implemented without any change.

In order to validate our analysis, we generate $t\bar{t}{H_{{}_{\text{SM}}}}$ events using MadGraph5 Alwall et al. (2011), and study each of the Higgs decay modes ($\tau^{+}\tau^{-}$, $WW^{*}$ and $ZZ^{*}$) independently. We match these events up to one additional jet and shower them with Pythia6 Sjostrand et al. (2006) using MLM matching Hoeche et al. (2005) with the $k_{T}$ shower scheme Alwall et al. (2014). As mentioned above, we use both PGS 4 Conway and Delphes 3 deF (2014) to simulate the detector response. The expected event yields obtained by our MC study for each of the three Higgs decay channels are given in tables 1, 2 and 3. We observe that Delphes and PGS exhibit complementary performances. Delphes gives a better modeling of the ATLAS efficiencies in the $2\ell 1\tau$ region, whereas PGS shows better agreement in the $2\ell 0\tau$, $3\ell$ and $4\ell$ regions. Overall, PGS provides a better modeling of the SM $t\bar{t}{H_{{}_{\text{SM}}}}$ efficiencies, and therefore we use PGS for recasting the results in ATL (2016d) in terms of rare top decays to $b\,H^{+}$. In our analysis, we use the recasting procedure as is. That is, we do not apply any ad hoc correction factors to the efficiencies, since we could not determine exactly the origin of our small discrepancies. The overall uncertainties in our detector simulation translate into a factor of $\sim 2$ uncertainty in our results for the signal yields.

A.3 Signal generation

A charged Higgs can be produced along with a top quark and a bottom quark through two distinct processes. First, as highlighted throughout this paper, this final state can arise from on-shell top pair-production, with one of the tops decaying to $H^{+}b$. As the charged Higgs gets heavier than about $155$ GeV, however, the branching ratio $\text{Br}(t\rightarrow b\,H^{+})$ gets suppressed, and off-shell production of the charged Higgs must be included as it gives an important contribution to the signal rate. The dominant non-resonant $H^{\pm}$ production process, $pp\rightarrow\bar{t}\,b\,H^{+}$, is shown in Fig. 8. We include this process in our study for all charged Higgs masses above $150$ GeV, and use the NLO cross sections provided in Degrande et al. (2016).

We use MadGraph Alwall et al. (2011) to generate $t\bar{t}+j$ events, and to decay both tops to $(\bar{b}W^{-})(b\,H^{+})$ (and the corresponding charge conjugate process). We match these events up to one additional jet and shower them using Pythia Sjostrand et al. (2006) with MLM matching Hoeche et al. (2005) and the $k_{T}$ shower scheme Alwall et al. (2014). We use Pythia to further decay the $W$’s, $H^{\pm}$’s and $A^{0}$’s. For the detector simulation, we use PGS Conway with the same settings as described above. The non-resonant process $tbH^{\pm}\rightarrow(\bar{b}W^{-})bH^{\pm}$ and its charge conjugate are generated with MadGraph. The subsequent steps are the same as for the first process.

Appendix B Signal yields and statistical procedure

B.1 Signal yields

As discussed in Appx. A, we use MC to obtain the signal efficiencies in each of the four signal regions defined by ATLAS, for both the $t\bar{t}$ on-shell process:

[TABLE]

as well as the non-resonant process in Fig. 8

[TABLE]

We generically denote the respective efficiencies by $\epsilon_{\text{res}}$ and $\epsilon_{\text{nonres}}$.

The signal yield for a specific signal region is then given by:

[TABLE]

where

[TABLE]

and

[TABLE]

For the total cross sections, we use \sigma_{t\bar{t}}\,\big{|}_{\text{13 TeV}}=830 pb, and the following approximate fit for $\sigma_{\text{nonres}}$ extracted from Degrande et al. (2016):

[TABLE]

in the range $m_{H^{\pm}}\sim[150-175]$ GeV.

B.2 Branching ratios

Throughout our study we set $\text{Br}(A^{0}\rightarrow\tau^{+}\tau^{-})=0.083$.

The top branching ratio to $b\,H^{+}$ is given by Gunion et al. (2000):

[TABLE]

where

[TABLE]

Above, $p_{H^{+}}$ is the momentum of $H^{+}$ in the top’s rest frame,

[TABLE]

and $p_{W^{+}}$ is defined in an analogous way.

The branching ratios of the charged Higgs are computed from its various widths. For completeness, we list all of them below.

The charged Higgs width to a pair of light on-shell SM fermions is given by:

[TABLE]

where $N_{c}=3$ and $U_{f\bar{f}^{\prime}}$ is an element of the CKM matrix if $f,\bar{f}^{\prime}$ are quarks, and $N_{c}=1$ and $U_{f\bar{f}^{\prime}}=1$ otherwise; $m_{f}$, $m_{\bar{f}^{\prime}}$ above are the running masses at $\mu=m_{H^{+}}$.

Since we are interested in spectra where $m_{H^{+}}<m_{t}$, the width $\Gamma(H^{+}\rightarrow t^{*}\,\bar{b})$ goes via an off-shell top quark, and is given by Djouadi et al. (1996):

[TABLE]

where $\kappa_{t}=m_{t}^{2}/m_{H^{+}}^{2}$, and $\kappa_{W}=m_{W}^{2}/m_{H^{+}}^{2}$. Above, we neglect the contribution from the bottom yukawa coupling.

The bosonic widths of the charged Higgs, $\Gamma(H^{+}\rightarrow W^{+(*)}\,\phi^{0})$, depend on whether the final state $W^{+}$ is on- or off-shell. In the former case, we have:

[TABLE]

where $\zeta_{\phi}$ for $\phi^{0}={H_{{}_{\text{SM}}}},\,H^{0},\,A^{0}$ are given in Sec. II, and

[TABLE]

On the other hand, if the $W^{+}$ is off-shell, we have Djouadi et al. (1996):

[TABLE]

where

[TABLE]

and

[TABLE]

B.3 Statistical treatment of fits and exclusions

For each signal region in ATL (2016d) with $N_{\text{sr}}$ observed events, $\mu_{B\,\text{sr}}\pm\Delta B_{\text{sr}}$ expected background events, and $S_{\text{sr}}$ expected signal events, we define the likelihood function:

[TABLE]

where $P$ and $G$ are Poisson and Gaussian distributions, respectively. The likelihood for the combination of all four signal regions is given by the product of the individual likelihoods:

[TABLE]

where $\theta_{B}\,=\,(B_{\text{sr1}},\,B_{\text{sr2}},\,B_{\text{sr3}},\,B_{\text{sr4}})$, and $\mu_{S}=(m_{H^{+}},m_{A^{0}},\rm tan\beta)$ uniquely specifies a point in the model parameter space, and unambiguously determines the signal yield $S_{\text{srj}}$ in each signal region. Since the correlations between background uncertainties in the four signal regions were not provided in ATL (2016d), we treat these uncertainties as uncorrelated. We note, however, that the post-fit backgrounds in ATL (2016d) did not change substantially relative to their pre-fit counterparts. This, combined with the factor of 2 uncertainties in our MC modeling of signal efficiencies, leads us to expect that the our results would not change substantially were we to properly include all background correlations in our fits.

For obtaining limits, we use a profiled log-likelihood analysis. First, we define:

[TABLE]

where the unconditional likelihood estimators $\hat{\mu}_{S},\,{\hat{\theta}}_{B}$ maximize the global $\text{log}\,\mathcal{L}(\mu_{S},\theta_{B})$, and the conditional estimator $\hat{\hat{\theta}}_{B}$ maximizes $\text{log}\,\mathcal{L}(\mu_{S},\theta_{B})$ for a given $\mu_{S}$. Since there are $3$ independent degrees of freedom in ${\mu}_{S}$, namely, $m_{H^{+}}$, $m_{A^{0}}$, and $\rm tan\beta$, the $p$-value for a specific model defined by ${\mu}_{S}$ is determined by:

[TABLE]

where $\text{CDF}(\chi^{2}_{3},\,-2\,\lambda(\mu_{S}))$ is the cumulative distribution function for a $\chi^{2}$-distribution with 3 degrees of freedom, evaluated at $-2\,\lambda(\mu_{S})$. From the $p$-value we can determine the exclusion confidence level for all points in the studied parameter space.

Finally, we note that when finding the goodness of fit of a given mass point $\mu^{\prime}_{S}=(m_{H^{+}},m_{A^{0}})$, we profile over the value of $\rm tan\beta$ that maximizes the log-likelihood, i.e., we use

[TABLE]

In this case, we define the $p$-value in an analogous way,

[TABLE]

but instead use a $\chi^{2}$-distribution with only 2 degrees of freedom.