Charged Higgs searches in the Georgi-Machacek model at the LHC

TL;DR

This paper investigates the constraints on the Georgi-Machacek model from LHC charged Higgs searches, showing current bounds limit the model's parameter space and proposing future detection channels with potential for high significance at the HL-LHC.

Contribution

It provides the first comprehensive analysis of LHC bounds on charged Higgs in the Georgi-Machacek model and proposes a new promising detection channel for future runs.

Findings

Current LHC bounds restrict the triplet vev in the model.

The combined theoretical and experimental constraints significantly limit the parameter space.

A new decay channel ($W^{b}$ + Higgs) can be probed with high significance at HL-LHC.

Abstract

The Georgi-Machacek Model allows for a large triplet vacuum expectation value (vev) with a custodial symmetric scalar potential. This manifestly leads to large charged Higgs coupling to SM fermions thus enhancing the detection probability of the model at the Large hadron collider. We show that the latest bound from LHC searches on singly charged Higgs scalar decaying to top and bottom quark restricts the triplet vev from above. We demonstrate that the combined limit from theoretical constraints and the latest Higgs data together with this charged Higgs bound can already curb the model parameter space significantly. Further, we propose the charged Higgs decaying to $W^{\pm}$ boson and a SM-like Higgs as a prospective channel to probe the yet unbounded model parameter space at the future run of the LHC. A simple cut-based analysis for this channel has been made showing the parameter region that can be probed with $5\sigma$ significance at the high luminosity LHC run.

Tables

Table 1. Table 1 : Benchmark points valid by all constraints and the corresponding production cross-section of our signal.

	$\sin \alpha$	$v_t$	$m_3$	$m_5$	${\sigma }_{prod}$
		(in GeV)	(in GeV)	(in GeV)	(in fb)
BP1	0.6	35	250	300	14.80
BP2	0.5	45	300	350	18.53

Table 2. Table 2 : No. of events obtained after each cut for both signal ( ℓ + ℓ − + 3 b + E T / \ell^{+}\ell^{-}+3b+E_{T}/\;\> ) and background and the significance obtained for 3 ​ ( 1 ) ​ a ​ b − 1 3 1 𝑎 superscript 𝑏 1 3(1)~{}ab^{-1} integrated luminosity at 14 ​ TeV 14 TeV \rm 14~{}TeV LHC.

	Effective cross-section after the cut(fb)			Significance reach at
SM-background	Preselection cuts	$E_T/$ cut	Invmass cut	3(1) $a{b}^{-1}$
$t\overline{t}+$jet	124.41	119.52	27.17	integrated
$t\overline{t}h$	1.05	0.97	0.32	luminosity
$t\overline{t}{W}^{\pm }$	0.21	0.19	0.049	at 14 TeV
$t\overline{t}Z$	0.84	0.79	0.21	LHC
$\mathrm{Total}\mathrm{SM}\mathrm{Background}$	126.51	121.49	27.75	–
$M_{H_{3^{\pm }}}=250\mathrm{GeV}$	1.49	1.42	0.52	5.37 (3.10)
$M_{H_{3^{\pm }}}=300\mathrm{GeV}$	1.87	1.78	0.65	6.68 (3.86)

Equations

$$V(\Phi ,X)$$

$$\displaystyle t_{1}=\frac{1}{\sqrt{2}}\left(\begin{array}[]{ccc}0&1&0\\ 1&0&1\\ 0&1&0\\ \end{array}\right)\,,\qquad t_{2}=\frac{1}{\sqrt{2}}\left(\begin{array}[]{ccc}0&-i&0\\ i&0&-i\\ 0&i&0\\ \end{array}\right)\,,\qquad t_{3}=\left(\begin{array}[]{ccc}1&0&0\\ 0&0&0\\ 0&0&-1\\ \end{array}\right)\,.$$

$$\displaystyle U=\frac{1}{\sqrt{2}}\left(\begin{array}[]{ccc}-1&0&1\\ -i&0&-i\\ 0&\sqrt{2}&0\\ \end{array}\right)\,.$$

$$\displaystyle\Phi=\left(\begin{array}[]{cc}\phi^{0*}&\phi^{+}\\ -\phi^{-}&\phi^{0}\\ \end{array}\right)\,,\qquad X=\left(\begin{array}[]{ccc}\chi^{0*}&\xi^{+}&\chi^{++}\\ -\chi^{-}&\xi^{0}&\chi^{+}\\ \chi^{--}&-\xi^{-}&\chi^{0}\\ \end{array}\right)\,.$$

$$\sqrt{v_d^2+8v_t^2}=v=246\mathrm{GeV}.$$

$${\mu }_{\varphi }^2$$

$${\mu }_X^2$$

$$H_5^{\pm }$$

$$h$$

$$H$$

$$\tan \beta =\frac{2\sqrt{2}{v}_t}{v_d}.$$

$${\lambda }_1$$

$${\lambda }_2$$

$${\lambda }_3$$

$${\lambda }_4$$

$${\lambda }_5$$

$${\kappa }_i=\frac{g_{hii(GM)}}{g_{hii(SM)}},i=f,W^{\pm },Z,\gamma .$$

$$pp\to t{H}_3^{-}\to b{W}^{+}{W}^{-}h\to 3b+2\ell +E_T/$$

$$p_T(j,b)$$

$$p_T(\ell )$$

Full text

1 Introduction

The Standard Model (SM) of particle physics has achieved its triumph after the discovery of a 125 GeV scalar resonance at the Large Hadron Collider (LHC) [1, 2] experiment. Nevertheless, the quest for new physics is yet to accomplish. Possibility of having an exotic particle in the realm of current collider reach still can not be discarded and it may just need a closer look at the present LHC data. In the second run of the LHC with increased energy and luminosity, a greater amount of data is already available for more detailed and intricate analysis.

Among many, a charged scalar particle is one of the appealing exotic candidates in particle phenomenology and has been searched for long. Most of the beyond Standard Model (BSM) scenarios render singly charged scalar candidate and it has always remained an important probe for the models with SM scalar sector extensions [3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]. In this paper, we will talk about probing the Georgi-Machacek (GM) model [16, 17, 18], a variant of scalar triplet extension of the SM, through the searches of a singly-charged scalar at the current and future LHC run. The usual Higgs triplet model (HTM) is favored for the explanation on the neutrino mass generation but it suffers from electroweak (EW) $\rho$-parameter constraint. The GM model, on the other hand, consists of two scalar triplets, one real and one complex, thus, preserving the custodial SU(2) symmetry and lifting the bound on the triplet vacuum expectation value (vev). This large triplet vev induces a large mixing between the GM and the SM sector leading to interesting search processes for the non standard scalars. In particular, the charged scalar of the GM model couples to the SM fermions with a strength directly proportional to the vev $(v_{t})$ and a significantly large $v_{t}$ enhances the detection possibility of such charged scalar at the collider. Such decay processes are highly suppressed in the HTM and can thus be an important probe of the GM model.

Besides the two CP-even scalars, the lightest of which resembles the SM-like Higgs with mass 125 GeV, there lies seven scalar particles in the GM particle spectrum arising from the custodial triplet and fiveplet respectively. The singly-charged scalar of the custodial fiveplet ($H_{5}^{\pm}$), however, has no tree-level interaction with the SM fermions. The singly charged scalar of the custodial triplet ($H_{3}^{\pm}$) can have significant tree level coupling to SM fermion-anti-fermion pair due to the large overlap with the Higgs field corresponding to the triplet vev. In particular, $H_{3}^{+}$ may acquire a considerable branching ratio (BR) to the $t\bar{b}$ channel in the kinematically possible mass region. Recently, the ATLAS collaboration [19] has reported their latest analysis on charged Higgs searches with the data from the 13 TeV run of the LHC with 36 $fb^{-1}$ luminosity and has provided a model independent bound on the production cross-section times the BR for charged Higgs decaying into $t\bar{b}$ mode. Following their analysis, in this paper, we aim to put constraints on the GM model parameter space in the physical basis in terms of the scalar masses and mixing angles. This limit will definitely put an upper bound on $v_{t}$ for a particular charged Higgs mass corresponding to its BR.

On the other hand, the theoretical constraints on the GM scalar potential from EW vacuum stability and perturbative unitarity at tree-level already limits $v_{t}$ from the lower end. Simultaneously, the charged scalar masses are also restricted from above due to the unitarity constraints. We show the bounds for two variants of GM potential, with and without the trilinear terms where the potential with vanishing trilinear term results in a more stringent bound. Additionally, the latest LHC Higgs data also cause a severe constraint in the model parameter space. Therefore, a combination of theoretical constraint, Higgs data and the limit from charged Higgs analysis by ATLAS will put a definitive restriction on the GM model. In this study, we have given a comprehensive description in this regard. Furthermore, as a future endeavor, we show how the region allowed by the above bounds can be probed at the future run of the LHC. For this, we propose the other competent decay channel of $H_{3}^{\pm}\to W^{\pm}h$ [5], $h$ being the 125 GeV Higgs resonance. This decay mode is dominant for large neutral scalar mixing angle $\alpha$ and can be an effective probe in the region where the ATLAS limit is relaxed. It should be mentioned that though the GM model have been studied extensively in the post-Higgs discovery era [20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32], implication of charged Higgs searches in association with the latest Higgs data to explore the model parameter space has not been discussed so far.

The paper is organized in the following manner. In sec. 2, we introduce the model briefly and explain the theoretical as well as constraint from Higgs data on the model parameter space. In sec. 3, we explain the production and decay of the charged Higgs state $H_{3}^{\pm}$ for various parameter space and portray the limits provided by the ATLAS analysis. In sec. 4, we demonstrate our analysis for the future probe of the model at the LHC and, in sec. 5 we show our findings collectively. Finally, we conclude in sec. 6.

2 The Model

Here, we present a brief description of the scalar potential of the GM model. in addition to the SM particle content, the scalar sector of the GM model [16, 17, 18] consists of one real $SU(2)_{L}$ triplet $\xi$ and one complex $SU(2)_{L}$ triplet $\chi$ with hypercharges $Y=0$ and $Y=2$ forming a bi-triplet $X$. Now, the most general scalar potential for the GM model can be written as [23, 30]

[TABLE]

where, $\tau_{a}\equiv\sigma_{a}/2$, ($a=1,2,3$) with $\sigma_{a}$’s being the Pauli matrices and $t_{a}$’s are the generators of triplet representation of $SU(2)$ which are expressed as

[TABLE]

Furthermore, the matrix $U$ in the trilinear terms of Eq. (1) is given by,

[TABLE]

It is worth mentioning here that the absence of these trilinear term would enhance the symmetry of the potential. Moreover, it should be noted that the GM potential has no explicit CP violation since there are no non-Hermitian terms.

Now, the bi-doublet $\Phi$ and the bi-triplet $X$ are represented as

[TABLE]

After the electroweak symmetry breaking (EWSB), the neutral components of the bi-doublet and the bi-triplet can be expanded around their vacuum expectation values(vev) as $\phi^{0}=\frac{1}{\sqrt{2}}(v_{d}+h_{d}+i\eta_{d})$, $\xi^{0}=(v_{t}+h_{\xi})$ and $\chi^{0}=(v_{t}+\frac{h_{\chi}+i\eta_{\chi}}{\sqrt{2}})$. The equality in vevs to the real and the complex triplets corresponds to the preserved custodial symmetry of the potential. The EW vev in terms of the vevs of the scalar multiplets turns out to be

[TABLE]

Thus, there will be two independent minimization conditions corresponding to the two vevs of the bi-doublet and the bi-triplet ($v_{d}$ and $v_{t}$) which can be used to extract the bilinear coefficients of the potential $\mu_{\phi}^{2}$ and $\mu_{X}^{2}$ as

[TABLE]

The scalar potential consists of a custodial quintuplet $(H_{5}^{++},H_{5}^{+},H_{5}^{0},H_{5}^{-},H_{5}^{--})$ of common mass $m_{5}$ and a custodial triplet $(H_{3}^{+},H_{0},H_{3}^{-})$ of mass $m_{3}$. Alongside, there are two CP-even scalars that are custodial singlets, namely $h$ and $H$ with masses $m_{h}$ and $m_{H}$ respectively. The neutral scalar mixing angle $\alpha$ diagonalizes the CP even sector to obtain the mass eigenstates $h$ and $H$. Hereby, we refrain ourselves from giving a detailed description of the diagonalization procedure and we refer the reader to Refs. [21, 23]. We only show the mass eigenstate for the charged scalars and the custodial singlets.

[TABLE]

where, $h_{d},h_{\chi},h_{\xi}$ are the fields corresponding to the neutral components of the doublet and the triplets after the electroweak symmetry breaking. The angle $\alpha$ represents the neutral mixing angle while $\tan\beta$ is defined as

[TABLE]

In total, there are nine independent parameters in the GM scalar potential with two bilinears ($\mu_{\phi}^{2}$ and $\mu_{X}^{2}$), five quartic couplings ($\lambda_{i}$, $i=1,\dots,5$) and two trilinear couplings ($M_{1}$ and $M_{2}$). Among these, the bilinears have already been traded in favor of the vevs, $v_{d}$ and $v_{t}$ as in Eq. (23). Besides the trilinear terms ($M_{1},M_{2}$), the five quartic couplings can now be exchanged with the four physical scalar masses, $m_{5}$, $m_{3}$, $m_{H}$ and $m_{h}$ and the mixing angle, $\alpha$. It should be mentioned here that, $h$ is the lightest among all the $CP$-even scalars corresponding to the Higgs-like scalar discovered at the LHC with mass $m_{h}\approx 125$ GeV. Below, we present the relation between the $\lambda_{i}$-s with the physical masses and mixings [30].

[TABLE]

2.1 Theoretical Constraint and Higgs data

In this section, we discuss the consequences of theoretical constraints, such as the electroweak stability and the perturbative unitarity on the GM scalar potential as well as the parameter constraints coming from the latest Higgs data. We will thus use Eq. (26) to translate the scalar couplings in terms of the physical masses and mixings.

We first consider the theoretical constraints on the model parameter space. In Fig. 1, we show the region allowed simultaneously by the electroweak vacuum stability and the perturbative unitarity bounds on the scalar quartic couplings [33, 23]. For illustration purpose, we have shown the allowed region in the $v_{t}$-$m_{3}$ plane and in the $m_{3}$-$m_{5}$ plane. In doing so, we choose two different possibilities for the GM scalar potential, with and without the trilinear couplings. For nonzero trilinear couplings, we choose $M_{1}=M_{2}=100~{}\rm GeV$ as a benchmark. As is evident from the plots that the theoretical constraints impose a lower bound on the triplet vev $v_{t}$ as well as an upper bound on the mass of the custodial triplet $m_{3}$ and custodial fiveplet $m_{5}$ when the scalar potential does not contain any trilinear terms, i.e. $M_{1}=M_{2}=0$. It also draws a correlation between $m_{3}$ and $m_{5}$ as can be seen from the lower panel of Fig. 1. These limits are actually a direct consequence of the unitarity condition on the scalar quartics as has been discussed in detail in Ref [30]. Briefly, the triplet is required to obtain a reasonable vev $v_{t}\gtrsim 27\rm GeV$ to get a simultaneous solution of all the unitarity conditions. The limits can however be lifted by the introduction of the non-zero trilinear terms in the potential as can be viewed from the right panel of Fig. 1. The reason is the additional contribution of the nonzero trilinear couplings to the scalar quartic couplings, given in Eq. (26), that helped to alleviate the bounds on the masses and the vev $v_{t}$. Moreover, with non-zero trilinear terms, the limit for small or vanishing $v_{t}$ can be achieved for large non standard scalar masses $m_{3}\simeq m_{5}\simeq m_{H}\gg m_{h}$ and for $M_{1}\simeq{\cal O}(v_{t})$, this limit is often referred to as the decoupling limit [23].

Next, one should also consider the bounds on the model parameter space placed by the updated 13 TeV LHC Higgs data [34, 35]. Here, it is to be mentioned that the lightest among all the CP-even states $(m_{h})$ corresponds to the 125 GeV SM-like Higgs in our model scenario. Therefore, its tree-level couplings to the SM fermions and vector bosons must be matched with the LHC data. Additionally, the charged scalar particles coming from both the custodial multiplets of masses $m_{3}$ and $m_{5}$ contributes to the loop-induced Higgs to diphoton decay mode. Therefore, the observed limit from $h\to\gamma\gamma$ decay [34, 35] should also be considered for lighter charged Higgs masses. In fact, the combined limits from Higgs decay to fermions, weak gauge bosons together with di-photon channel severely constraint the model parameter space as shown in Fig. 2. To implement the current limit, we define the Higgs coupling modifiers as the ratio of the theoretical model prediction to the SM value, i.e,

[TABLE]

The model predictions for the coupling modifiers are well-known [30] and it depend on the mixing angles $\alpha$ and $\beta$ for a fixed $m_{3}$ and $m_{5}$. In Fig. 2, we show the allowed region in the $(\sin\alpha-v_{t})$ plane from the latest 13 TeV LHC Higgs result [34, 35]. The blue shaded region in the figure is allowed by all Higgs channels including the loop induced diphoton mode. For the diphoton decay mode, we have fixed the charged scalar masses at $m_{3}=300~{}\rm GeV$ and $m_{5}=400~{}\rm GeV$. The benchmark of such mass points are followed by the maximum allowed value by the theoretical constraints, as is evident from Fig. 1. Now, in Fig. 2, the solid(red) and dashed(black) lines further corresponds to the equivalent theoretical limits on $v_{t}>35(10)~{}\rm GeV$ for the two cases, $M_{1}=M_{2}=0(100~{}\rm GeV)$ respectively with $m_{3}=300~{}\rm GeV$ following the upper panel of Fig.1. Obviously, the limit is more relaxed in the non vanishing trilinear case, allowing $v_{t}$ as low as around 10 GeV. The most important point in Fig. 2, is however, the region allowed by the Higgs data. It shows a clear preference towards larger $\sin\alpha$ at large $v_{t}$ allowing only $\sin\alpha>0.3$ for $v_{t}\gtrsim 30~{}\rm GeV$. This has notable consequences on the charged Higgs decay modes as we will explain in our subsequent analysis.

3 Charged Higgs production and decay

As mentioned earlier, the GM model contains two pairs of singly charged Higsses, one from each of the custodial triplet $(H_{3}^{\pm})$ and the custodial quintuplet $(H_{5}^{\pm})$. However, the quintuplet component $H_{5}^{\pm}$ does not couple to SM quarks since it has a pure triplet origin, as can also be seen from Eq. (24a). On the other hand, the triplet candidate $H_{3}^{\pm}$ can mix with SM quarks through doublet mixing. Therefore, in this article, we will venture the possibility of limiting or probing the GM model from direct searches of the triplet charged candidate, namely, $H_{3}^{\pm}$. To begin with, we show the decay branching ratio of this particle for various mass regions. In Fig. 3, we depict the BR of $(H_{3}^{\pm})$ of mass $m_{3}$ for two different choices of $v_{t}=(30,45)~{}\rm GeV$ and each for four different values of $\sin\alpha=(0.001,0.01,0.1,1.0))$, represented by four different color variations. It must be mentioned that our choice of $v_{t}$ relies on the parameter space available after satisfying the theoretical constraints, as described in Sec. 2.1. Although, the non-zero trilinear scenario allows much lower $v_{t}$, the BRs has no dependence on the parameter values $M_{1}$ and $M_{2}$. Therefore, in our following analysis, we will mainly focus on $v_{t}\gtrsim 30~{}\rm GeV$. It should however, be mentioned that we have not imposed the Higgs data while plotting Figs. 3, 4 and 5.

Let us first briefly understand the nature of Fig. 3. As one can see, $H_{3}^{\pm}$ has two dominant decay channels, $H_{3}^{+}\to t\bar{b}$ and $H_{3}^{+}\to W^{+}h$ (similarly for the anti-particle), $h$ being the 125 GeV resonance. It is clear from Fig. 3, that as the mass of $H_{3}$ reaches 205 GeV (threshold to produce $W^{\pm}h$), the $BR(H_{3}^{\pm}\rightarrow W^{\pm}h)$ increases considerably with an increase in $\sin\alpha$ reducing the $BR(H_{3}^{\pm}\rightarrow tb)$. This turns out to be the crucial point for the charged Higgs searches in this model. As we showed in Sec. 2.1 that the current Higgs data preferred larger $\sin\alpha$ for larger $v_{t}$. Hence, the combined limit of theoretical constraints with the Higgs data points towards an inclination of a lighter charged Higgs mass $m_{3}<300~{}\rm GeV$ and a larger mixing angle in the CP-even sector. From this viewpoint and referring to Fig. 3, one can interpret that probing the charged Higgs candidate in this model would be more probable in $W^{\pm}h$ channel instead of the quark final state.

Nevertheless, it is mandatory to validate the model parameter space first with the current limit on charged Higgs decaying to $tb$ final state, as reported by the ATLAS collaboration [19] with the 13 TeV data. There is a similar analysis by CMS collaboration lately [36], however there limits for low charged Higgs mass $m_{3}<500~{}\rm GeV$ are weaker and hence we will only consider the ATLAS bound. In Fig. 4, we draw the limits on the charged Higgs production cross sec times its BR to $tb$ channel, as observed in experiment (further details on cross section measurement are given in section 4). The observed limit on the effective cross section from ATLAS for a range of charged Higgs masses has been shown for three distinct triplet vev $v_{t}=30,~{}45~{}\&~{}60~{}\rm GeV$ with two different $\sin\alpha\equiv(0.1,1)$ values. As anticipated, even for low charged Higgs masses $m_{3}\lesssim 300~{}\rm GeV$, the null result from observed data can only exclude $v_{t}>45~{}\rm GeV$ for $\sin\alpha=0.1$. The limit is much relaxed for higher mixing angles.

Following the above findings, in the next section we will carry out a simple signal-to-background (cut-based) analysis to predict the efficiency of probing the model parameter space still allowed by theoretical constraints and the present experimental observation made for both the Higgs and the charged Higgs.

4 Analysis

In this section, we address the question that we posed in the last section, i.e. how efficiently we can probe the parameter space of GM model allowed by the experimental and theoretical bounds together. Charged Higgs decaying into $W^{+}h$ channel is the signal that we will be interested in. The single production of the charged Higgs can however be made either in the 4-flavor or in the 5-flavor scheme, where the production processes will be $pp\to t\bar{b}H_{3}^{+}$ and $pp\to tH_{3}^{+}$ respectively. In our subsequent analysis, we will work in the 5-flavor scheme.

In Fig. 5, we show the production cross section times the BR for $\sin\alpha=1,0.1$. Following our previous line of argument, a direct comparison between Fig. 4 and Fig. 5 shows that for larger $\sin\alpha$, the $W^{+}h$ decay mode for the charged higgs imparts larger effective cross-section. Hence, we may expect to discover or restrict the model in its full parameter region with larger efficiency in the future run of the LHC probing the specified channel. In the following, we will give a simple cut-based analysis to give a prediction of probing the GM model at the future LHC run from the charged Higgs searches through $W^{+}h$ final state. Therefore, we consider the following signal channel:

[TABLE]

where, we assume that the two $W$ bosons decays leptonically and the Higgs decay yields two b-tagged jets. Thus as a final state we look for 3 b-tagged jets, two opposite signed leptons and missing transverse energy. The irreducible SM background contribution will come from $t\bar{t}j$ with at least one extra hard jet, $t\bar{t}h$ and $t\bar{t}V,V={W^{\pm},Z}$ processes. It should be noted here that the $t\bar{t}$ plus jets background has the largest cross section among all and in turn it is the most dominant background in this analysis. Below we present our cut-based analysis for the signal-to-background event ratio computation. For our analysis, we choose two benchmark points in consistent with all the above limits, and the relevant parameter values are shown in Table. 1. In passing, we would like to mention that our benchmark value for $m_{5}$ is consistent with the latest bound from ATLAS collaboration [37] where a doubly-charged Higgs mass between 200 and 220 GeV is excluded at 95% CL from same-sign $W$ boson signal search.

The model has been implemented in FeynRules [38] to obtain the UFO model files required for event generation in madgraph. We generate both the signal and SM backgrounds events at the Leading Order (LO) in Madgraph5(v2.3.3) [39] using the NNPDF3.0 parton distributions [40]. The parton showering and hadronisation is done using the built-in Pythia [41] in the Madgraph. The showered events are then passed through Delphes(v3) [42] for the detector simulation where the jets are constructed using the anti-$K_{T}$ jet algorithm with minimum jet formation radius $\Delta R=0.4$. The isolated leptons are considered to be separated from the jets and other leptons by $\Delta R_{\ell i}\gtrsim 0.4,i=j,\ell$. For the background processes with hard jets, proper MLM matching scheme [43] has been chosen.

To generate our signal and background events, we employ the following pre-selection cuts.

[TABLE]

The $b$-jets are tagged with the $p_{T}$-dependent $b$-tag efficiency following the medium criteria of Ref. [44] which has an average 75% tagging efficiency of the $b$-jets with $50~{}{\rm GeV}<p_{T}<200~{}{\rm GeV}$ and 1% mis-tagging efficiency for light jets.

Additionally, we propose the following selection cuts to dis-entangle the signal from the SM backgrounds that would enhance the signal significance.

• •

**Selecting jets and leptons:**To affirm that our signal has 3 $b$ jets, we select 3 $b$ jets with $p_{T}(b)>$ 30 GeV while rejecting any fourth $b$ jet with $p_{T}>$ 35 GeV. In addition to this, we demand two opposite sign leptons with $p_{T}(\ell)>$ 10 GeV.

• •

Selecting $E_{T}/\;\>$: Both for signal and backgrounds, the missing energy($E_{T}/\;\>$) comes from the decay of $W^{\pm}$ boson. From the left panel of Fig. 6, one can see that the distributions for signal and backgrounds are peaking around similar $E_{T}/\;\>$ value. However, the SM background has a large tail of the distribution. We, thus, impose an upper bound of $E_{T}/\;\><175$ GeV as an optimum cut to enhance the signal over background ratio.

• •

Higgs mass reconstruction: In our signal, two $b$-tagged jets among the three are produced by the decay of the Higgs. Therefore, the invariant mass of the two $b$-jets are expected to peak around 125 GeV. On the other hand, only the $t\bar{t}h$ can yield similar peak structure for the $M_{b\bar{b}}$. The right panel of Fig. 6 depicts the nature of $M_{b\bar{b}}$ distribution for both the signal and background. It is to be mentioned that the pair of b-jets is selected after rejecting any third b-jet that has the minimum azimuthal angle $\phi$. This follows from the argument that the third $b$-jet is a remnant of direct top decay in our signal and therefore expected to lie closer to the beam direction than the other two. Following Fig. 6, we select $100~{}\rm GeV<M_{b\bar{b}}<150~{}\rm GeV$ for significant background reduction. This cut turns out to be the most effective to reduce the large $t\bar{t}j$ background events.

The cut-flow is presented in Table. 2 for our two chosen benchmarks (BP1, BP2) of Table. 1. The signal significance is defined as $\mathcal{S}=\frac{S}{\sqrt{S+B}}$, $S$ and $B$ being the number of signal and background events after all the cuts respectively.

As is evident from Table. 2, one can reach more than $5\sigma$ significance at the High Luminosity run of LHC (HL-LHC) with 3000 $fb^{-1}$ luminosity and more than $3\sigma$ significance with only 1000 $fb^{-1}$ in the specified parameter space of this model with this particular signal channel.

It is worth mentioning here that we did not consider the systematic uncertainty in background prediction while calculating the signal significance. At the LHC Run-II, the current systematic uncertainty on our dominant SM background process $t\bar{t}+\rm{jet}$ can be considered to be around 10% [35]. Inclusion of this uncertainty appropriately[45] yield the signal significance for our becnhmark points for $M_{H_{3^{\pm}}}=250(300)~{}\rm GeV$ respectively as 5.16(2.98)$\sigma$ and 6.46(3.73)$\sigma$ at 3(1)$\rm{ab^{-1}}$ luminosity. Therefore the inclusion of uncertainty will only make $\sim$4% deviation from our quoted significance reach given in Table. 2. In fact, even within a conservative approximation of 20% systematic uncertainty, the significance reach for our first signal point remains around $\approx 5\sigma$. Moreover, it should also be noted that the actual systematic uncertainty with the future high luminosity run of LHC at $\rm 14~{}TeV$ is currently unknown. Considering the future systematic uncertainty to be much lower than the current value [46], our prediction on signal significance will not differ much from the realistic result and is useful for an preliminary understanding.

5 Results

Let us now present our findings in a comprehensive manner. In Fig. 7, we show our complete results spanned in the model parameters $\{\sin\alpha-v_{t}\}$ plane for two particular choices of $m_{3}$ and $m_{5}$ masses corresponding to our benchmark points in Table 1. The region allowed by the latest LHC Higgs data [34, 35] and the bounds from the theoretical constraints for the two distinct cases of trilinear terms follows the same color combinations as in Fig. 2. On top of it, we show the bounds imposed by the ATLAS analysis for 13 TeV data on charged Higgs searches in $tb$ channel [19] which is shown by the gray shaded region bounded by the solid black line. As can be seen from the Fig. 2, this bound can only restrict a small parameter space in association with the Higgs data. Quantitatively, for charged Higgs masses around 250 GeV, $v_{t}>50~{}\rm GeV$ can only be discarded. For such $v_{t}$, the bound on $\sin\alpha$ though comes solely from the Higgs data. As already mentioned, for smaller $v_{t}$, Higgs data only allows large $\sin\alpha$ which can only be probed by the other significant decay mode of the charged scalar to $Wh$ channel. Hence, we present the $3\sigma$ and $5\sigma$ discovery reach at the HL-LHC with 3 $ab^{-1}$ luminosity from our proposed signal channel for charged Higgs decaying to $W^{\pm}~{}h$ mode, depicted as solid (orange) and dashed (green) lines respectively. The areas above those lines shaded in Orange and Green describe the parameter space that can be probed at the future HL-LHC with more than $3\sigma$ and $5\sigma$ signal significance respectively. To illustrate further, one can see that a 5$\sigma$ discovery is possible for $v_{t}\gtrsim$ 31(35) GeV for $m_{3}=$250(300) GeV with 3 $ab^{-1}$ integrated luminosity. In a nutshell, Fig. 7 shows the complete allowed parameter space of GM model from theoretical constraints, latest LHC Higgs data and charged Higgs searches altogether, while simultaneously showing the future reach of charged Higgs searches for the unbounded parameter space.

Before concluding, we should remark on the limit imposed by the CMS doubly charged Higgs search analysis at 13 TeV LHC with 35 $\rm fb^{-1}$ luminosity [47]. The 95% C.L. upper limit has been given on the GM model based on the searches made for Vector Boson fusion production of a doubly charged Higgs decaying into a pair of same-sign W boson. For a 200-300 $\rm GeV$ doubly charged Higgs, the limit is translated as an exclusion of parameter space above $v_{t}>30$ GeV at 95% C.L. This already is quite in tension with the model scenario with vanishing trilinear terms ($M_{1}=M_{2}=0$) where the theoretical bounds only allow $v_{t}>30$ GeV. Therefore, following our prediction on charged Higgs decaying to $Wh$ mode in the future high luminosity run, a non-observance of such a signal would definitely seal the fate of the model scenario for $M_{1}=M_{2}=0$.

6 Summary

To summarise, we consider the GM model as a preferred variant of scalar triplet extension of the SM that preserves the custodial symmetry allowing for a larger triplet vev unlike the usual HTMs. Such large triplet vev leads to interesting search prospects for the singly charged scalar of the model that can decay to SM fermion with large coupling. The latest LHC search result on charged Higgs decaying to $tb$ channel thus impose an upper bound on the triplet vev. On the other hand, the theoretical constraints on the model parameter restricts the triplet vev from the lower end and can be as strong as 30 GeV for vanishing triliinear term of the GM potential. Moreover, the theoretical constraints entails an upper bound on the triplet and quintuplet masses which puts significant upper bound on the singly and doubly charged Higgs mass. Additionally, the non-standard triplet scalars mixing will modify the SM-like Higgs coupling to the SM fermions and gauge bosons. Therefore, it is mandatory to match the model prediction for the SM-like Higgs coupling with the latest LHC Higgs data. In the first part of this paper, we show the complete limit on the model parameter space from both the theoretical constraints as well as the 13 TeV LHC Higgs results. We show that these two constraints together put a stringent bound on the model parameter space in neutral mixing angle $\sin\alpha$ and triplet vev $v_{t}$ plane. As shown, a large positive $\sin\alpha\gtrsim 0.1(0.3)$ with $v_{t}>10(30)~{}\rm GeV$ is preferred by these bounds for the two distinct cases of vanishing and non-vanishing trilinear terms. In the second part of our paper, we intended to combine these limits with the latest bound from LHC on the charged scalar searches to $tb$ channel. We show that this limit excludes $v_{t}>50$ GeV for $\sin\alpha\lesssim 0.4$ for a 250 GeV charged Higgs in the region allowed by the theoretical constraints and the Higgs data. We also described that for a larger $\sin\alpha$, the singly charged Higgs predominantly decays to $Wh$ channel. Therefore, we showed a simple cut-based signal-to-background analysis for the singly charged Higgs decaying to $Wh$ mode. Our analysis revealed that this search channel can probe the unbounded region of the model parameter by all the above-mentioned constraints at $5\sigma$ signal significance at the future high luminosity run of the LHC.

Acknowledgements

IS would like to thank Michihisa Takeuchi and Satoshi Shirai for useful discussions. The work of IS was supported by World Premier International Research Center Initiative (WPI), MEXT, Japan. NG would like to acknowledge the Council of Scientific and Industrial Research (CSIR), Government of India for financial support. SG would like to thank the University Grants Commission, Government of India, for a research fellowship.