A variant two-Higgs doublet model with a new Abelian gauge symmetry

TL;DR

This paper proposes a two-Higgs doublet model with a new Abelian gauge symmetry that explains the muon g-2 anomaly, provides a dark matter candidate, and remains consistent with flavor and collider constraints.

Contribution

It introduces a novel variant of the two-Higgs doublet model with a broken Abelian gauge symmetry, addressing muon g-2 and dark matter in a unified framework.

Findings

The model can explain the muon anomalous magnetic moment.

Dark matter interacts via Higgs portal and Z' boson.

Strong constraints from dark matter direct detection.

Abstract

We consider a two-Higgs doublet model extended with a broken Abelian gauge symmetry under which all Standard Model (SM) quarks, fourth generation fermions and a new SM-singlet scalar boson are charged. Such a setup is shown to be able to accommodate the muon anomalous magnetic dipole moment while being consistent with existing constraints of flavor-violating decays of charged leptons and Z boson. The new scalar boson offers a suitable dark matter candidate that interacts with the SM particles via the Higgs portal and the $Z'$ boson associated with the new gauge symmetry. The dark matter direct detection bound is found to impose a strong constraint on the new gauge coupling.

Tables

Table 1. Table 1: Quantum numbers of SM fermions in one generation and two Higgs doublets.

	SM fermions						Higgs
	$Q_L$	$u_R$	$d_R$	$L_L$	$e_R$	${\nu }_R$	${\Phi }_1$	${\Phi }_2$
$SU{\left(3\right)}_C$	3	3	3	1	1	1	1	1
$SU{\left(2\right)}_L$	2	1	1	2	1	1	2	2
$U{\left(1\right)}_Y$	1/6	2/3	$-1/3$	$-1/2$	$-1$	0	1/2	1/2
$U{\left(1\right)}_X$	$x_q$	$x_q$	$x_q$	0	0	0	0	0
$Z_2$	+	+	$-$	+	$-$	+	+	$-$

Table 2. Table 2: Quantum numbers of fourth-generation fermions and new scalar.

	Fourth generation fermions						Scalar
	$Q_{L4}$	$u_{R4}$	$d_{R4}$	$L_{R4}$	$e_{L4}$	${\nu }_{L4}$	$\varphi$
$SU{\left(3\right)}_C$	3	3	3	1	1	1	1
$SU{\left(2\right)}_L$	2	1	1	2	1	1	1
$U{\left(1\right)}_Y$	$1/6$	$2/3$	$-1/3$	$-1/2$	$-1$	$0$	$0$
$U{\left(1\right)}_X$	$x_{q4}$	$x_{q4}$	$x_{q4}$	$x_{\ell 4}$	$x_{\ell 4}$	$x_{\ell 4}$	$x_{\varphi }$
$Z_2$	$+$	$+$	$-$	$+$	$-$	$+$	$+$

Equations

$$\begin{array}{r}{Q}_{L4}\equiv {\left(\genfrac{}{}{0ex}{}{u}{d}\right)}_{L4},u_{R4},d_{R4},\\ L_{R4}\equiv {\left(\genfrac{}{}{0ex}{}{\nu }{e}\right)}_{R4},e_{L4},{\nu }_{L4},\end{array}$$

$$\begin{array}{rl}{\mathcal{L}}_{\text{Y}}=& -\left[f_{4i}^L{\bar{L}}_{R4}{L}_{Li}+f_{4i}^e{\bar{e}}_{L4}{e}_{Ri}+f_{4i}^{\nu }{\bar{\nu }}_{L4}{\nu }_{Ri}\right]\varphi \\ & +h.c.,\end{array}$$

$$f_{4i}^L=f_{4i}^e=f_{4i}^{\nu }\equiv f_{4i}$$

$${\Phi }_i=\left(\begin{array}{c}{\varphi }_i^{+}\\ (v_i+{\rho }_i+i{\eta }_i)/\sqrt{2}\end{array}\right),$$

$$h$$

$$H$$

$$V_{\text{2HDM}}$$

$$+\frac{{\lambda }_1}{2}{\left|{\Phi }_1\right|}^4+\frac{{\lambda }_2}{2}{\left|{\Phi }_2\right|}^4+{\lambda }_3{\left|{\Phi }_1\right|}^2{\left|{\Phi }_2\right|}^2+{\lambda }_4{\left|{\Phi }_1^{†}{\Phi }_2\right|}^2+\frac{{\lambda }_5}{2}[({\Phi }_1^{†}{\Phi }_2{)}^2+({\Phi }_1{\Phi }_2^{†}{)}^2],$$

$$V_{\varphi }$$

$$\begin{array}{rl}{m}_{\varphi }^2& =m_0^2+\frac{1}{2}{\kappa }_1{v}^2{\cos }^2\beta +\frac{1}{2}{\kappa }_2{v}^2{\sin }^2\beta ,\\ {\lambda }_{h}v& =(-{\kappa }_1\sin \alpha \cos \beta +{\kappa }_2\cos \alpha \sin \beta )v,\\ {\lambda }_{H}v& =({\kappa }_1\cos \alpha \cos \beta +{\kappa }_2\sin \alpha \sin \beta )v.\end{array}$$

$${\lambda }_h\simeq {\kappa }_2\frac{\sin \beta }{\cos \alpha }.$$

$$\begin{array}{rl}{y}_u^h& =\frac{\cos \alpha }{\sin \beta }=\cos (\beta -\alpha )\cot \beta +\sin (\beta -\alpha ),\\ y_d^h& =-\frac{\sin \alpha }{\cos \beta }=\sin (\beta -\alpha )-\cos (\beta -\alpha )\tan \beta .\end{array}$$

$$m_{u_4}$$

$$m_{{\nu }_4}$$

$$m_{H^{+}}$$

$$\Delta a_{\mu }=\frac{{\left|f_{4\mu }\right|}^2}{8{\pi }^2}{\int }_0^{1}dx\frac{x^2[m_{\mu }^2(1-x)+m_{\mu }{m}_{e_4}]}{(1-x)(m_{\varphi }^2-m_{\mu }^{2}x)+m_{e_4}^{2}x}.$$

$$\begin{array}{rl}& \text{BR}({\mu }^{-}\to e^{-}\gamma )\simeq \frac{\Gamma (\mu \to e\gamma )}{\Gamma (\mu \to e\nu \bar{\nu })}\\ & =\frac{m_{\mu }^3}{16\pi }\left({\left|{\sigma }_L\right|}^2+{\left|{\sigma }_R\right|}^2\right)\frac{192{\pi }^3}{G_F^2{m}_{\mu }^5},\end{array}$$

$$\begin{array}{rl}& {\sigma }_L={\sigma }_R\\ & \simeq i(4\pi {)}^{2}e{f}_{4e}^{\ast }{f}_{4\mu }{\int }_0^{1}dz{\int }_0^{1-z}dy\frac{m_{e_4}(z-1)}{m_{\varphi }^{2}z+m_{e_4}^2(1-z)}.\end{array}$$

$$\text{BR}({\mu }^{-}\to e^{-}\gamma )$$

$$\text{BR}({\tau }^{-}\to e^{-}\gamma )$$

$$\text{BR}({\tau }^{-}\to {\mu }^{-}\gamma )$$

$$\text{BR}({\mu }^{-}\to e^{-}{e}^{-}{e}^{+})$$

$$\text{BR}({\mu }^{-}\to e^{-}{e}^{-}{e}^{+})/\text{BR}({\mu }^{-}\to e^{-}\gamma )\approx 0.006.$$

$$\begin{array}{rl}& \text{BR}(Z\to {\ell }_i^{\pm }{\ell }_j^{\mp })\\ & =\frac{m_Z}{12\pi {\Gamma }_Z}\left[2\left(f_V^2+f_A^2\right)+\left(f_M^2+f_E^2\right)m_Z^2\right],\end{array}$$

$$\begin{array}{rl}{f}_V=& \left[g^{\prime }\left(-\frac{1}{4}+{\sin }^2{\theta }_W\right)\right]\frac{f_{4j}{f}_{4i}^{\ast }}{(4\pi {)}^2}\left[I_1+I_2\right],\\ f_A=& \frac{g^{\prime }}{4}\frac{f_{4j}{f}_{4i}^{\ast }}{(4\pi {)}^2}\left[I_1+I_2\right],\\ f_M=& \left[g^{\prime }\left(-\frac{1}{4}+{\sin }^2{\theta }_W\right)\right]\frac{f_{4j}{f}_{4i}^{\ast }}{(4\pi {)}^2}{I}_1,\\ f_E=& \frac{g^{\prime }}{4}\frac{f_{4j}{f}_{4i}^{\ast }}{(4\pi {)}^2}{I}_1,\end{array}$$

$$\begin{array}{rl}{I}_1& \equiv {\int }_0^{1}dz{\int }_0^{1-z}dy\frac{m_4^2+y(1-y-z)m_Z^2}{-y(1-y-z)m_Z^2+(1-z)m_4^2+z{m}_{\varphi }^2},\\ I_2& \equiv {\int }_0^{1}dzz\ln \left[z{m}_4^2+(1-z)m_{\varphi }^2\right]-{\int }_0^{1}dz{\int }_0^{1-z}dy\ln \left[(1-z)m_4^2+z{m}_{\varphi }^2-y(1-y-z)m_Z^2\right].\end{array}$$

$$\text{BR}(Z\to e^{\pm }{\mu }^{\mp })<1.7\times 10^{-6}$$

$$\text{BR}(Z\to e^{\pm }{\tau }^{\mp })<9.8\times 10^{-6}$$

$$\text{BR}(Z\to {\mu }^{\pm }{\tau }^{\mp })<1.2\times 10^{-5}$$

$${\sigma }_{\varphi p(n)}^{\mathrm{SI}}={\sigma }_{\varphi p(n)}^{\mathrm{SI}-h}+{\sigma }_{\varphi p(n)}^{\mathrm{SI}-Z^{\prime }},$$

$${\sigma }_{\varphi p(n)}^{\mathrm{SI}-h}=\frac{m_{p(n)}^2}{4\pi {\left(m_{\varphi }+m_{p(n)}\right)}^2}{\left(f_S^{p(n)}\right)}^2,$$

$$\frac{f_S^{p(n)}}{m_p}=\sum _{q=u,d,s}\frac{{\alpha }_q^S}{m_q}{f}_{Tq}^{p(n)}+\frac{2}{27}{f}_{TG}^{p(n)}\sum _{q=c,b,t}\frac{{\alpha }_q^S}{m_q}.$$

$$\begin{array}{rl}{f}_{Tu}^p& =0.0208\pm 0.0015,f_{Tu}^n=0.0189\pm 0.0014,\\ f_{Td}^p& =0.0411\pm 0.0028,f_{Td}^n=0.0451\pm 0.0027,\\ f_{Ts}^p& =0.043\pm 0.011,f_{Ts}^n=0.043\pm 0.011.\end{array}$$

$${\sigma }_{\varphi p(n)}^{\mathrm{SI}-Z^{\prime }}=\frac{(3x_q{)}^2{x}_{\varphi }^2{e}^{\prime 4}{m}_{\varphi }^2{m}_{p(n)}^2}{\pi {\left(m_{\varphi }+m_{p(n)}\right)}^2{m}_{Z^{\prime }}^4},$$

$${\mathcal{L}}_{\mu \to 3e}$$

Full text

I Introduction

Despite its great success in explaining almost all the existing observational data, the Standard Model (SM) is widely believed to be just an effective theory of a more fundamental theory. Even with the constraints of known symmetries and experimental data, there are still a plethora possible ways to extend the SM. Particularly interesting ones are those that can accommodate some of the outstanding anomalies and/or address certain theoretical issues.

One trivial extension of the SM is to replicate another generation of fermions, the sequential fourth generation. Besides its simplicity, the existence of fourth generation fermions may address some unanswered questions to the SM, such as electroweak symmetry breaking and bayon-antibaryon asymmetry Holdom:1986rn ; Hill:1990ge ; Hung:2010xh ; Hou:2011df ; Fok:2008yg . However, this possibility is excluded due to the conflict between the mass lower bound ($\gtrsim 680$ GeV) given by the LHC direct search for sequential fourth generation quarks Chatrchyan:2012fp and the mass upper bound ($\lesssim 550~{}{\rm GeV}$) required by the unitarity of the partial wave amplitude in fourth-generation top quark pair ($t^{\prime}\overline{t^{\prime}}$) scattering Erler:2010sk . Furthermore, the existence of sequential fourth generation fermions is also ruled out by the current Higgs boson data at the LHC, as an enhancement by a factor of $9$ or so in the Higgs boson production via gluon-gluon fusion (ggF) is expected due to the additional contributions of fourth-generation top and bottom quarks ($b^{\prime}$).

It has been recently shown that in the Type-II two-Higgs doublet model (2HDM) with sequential fourth generation fermions, both the Higgs data and the direct search at the LHC can be reconciled provided a subtle cancellation occurs in the Higgs ggF production Das:2017mnu . In certain parameter space, dubbed the wrong-sign scenario, the Yukawa coupling of $b^{\prime}$ can have an opposite sign to its up-type partner $t^{\prime}$ while they are almost degenerate in mass. As a result, their contributions to the Higgs ggF production amplitude cancel with each other.

Anomalous magnetic dipole moments of electron and muon have been provided as tests for the SM, as they can be both experimentally measured and theoretical computed to an extremely high precision. Interestingly, the muon anomalous magnetic dipole moment, $a_{\mu}\equiv(g-2)_{\mu}/2$, has been observed to be $\sim 3\sigma$ away from the SM prediction since its first measurement. This longstanding discrepancy, known as the muon $g-2$ anomaly, triggers many proposals beyond the SM (see, e.g., Refs. Jegerlehner:2009ry and Blum:2013xva for reviews).

Currently the experimental result shows that the discrepancy $\Delta a_{\mu}\equiv a_{\mu}^{\text{exp}}-a_{\mu}^{\text{SM}}=(27.05\pm 7.26)\times 10^{-10}$ reaches about $3.7\sigma$ Keshavarzi:2018mgv . With new data from the Fermilab experiment E989 in the near future, the significance is expected to be over $5\sigma$, assuming the central values of both theory and experiment results remain unchanged Blum:2013xva . If this is the case, the muon $g-2$ anomaly certainly involves contributions from beyond the SM. In Type-II 2HDM, additional contributions to $a_{\mu}$ have been calculated up to two loops, and the result is not large enough to explain the difference, even with the contributions of sequential fourth generation fermions Huo:2003pw ; Cheung:2003pw ; Broggio:2014mna .

In this paper, we point out that with the introduction of a new scalar boson having lepton flavor-violating couplings, the one-loop diagram involving a heavy chiral lepton can be enhanced. We will focus on the wrong-sign scenario in the type-II 2HDM containing fourth generation fermions, where the heavy charged chiral leptons are not canonically charged under the SM electroweak gauge group while the quarks stay sequential. The new scalar $\phi$ is assumed to be an SM singlet. It enhances the muon $g-2$ through its flavor-violating coupling between muon and the fourth-generation charged lepton $e_{4}$.

To avoid $\phi$ from having other undesirable couplings with fermions, we introduce a gauged $U(1)_{X}$ symmetry. Under the new Abelian symmetry, all fermions except for the leptons in the first three generations have nonzero charges. The two Higgs doublets are also neutral under $U(1)_{X}$. Assumed to be lighter than the fourth generation fermions, the $\phi$ field in the model offers a suitable dark matter (DM) candidate to explain the observed relic density. Moreover, the DM direct search bound imposes a strong constraint on the gauge coupling associated with the new $U(1)_{X}$ symmetry.

The structure of this paper is organized as follows. We start with the model setup in Section II. We present in Section III how the main contribution from the heavy charged lepton fills the gap between the experimental data and SM prediction for the muon $g-2$. We also show the constraints on the model parameter space due to the lepton flavor-violating decays of charged leptons and the $Z$ boson. In Section IV, we calculate the DM relic density and derive a bound from the direct search. Finally, conclusions are given in Section V.

II Model setup

We consider an extension of the usual type-II 2HDM with three right-handed neutrinos ($\nu_{R}$), a fourth generation of fermions:

[TABLE]

and a SM singlet scalar $\phi$. The model also has a new gauged $U(1)_{X}$ symmetry under which all but the leptons in the first three generations and the two Higgs doublets are charged. As a result, the heavy fermions $t^{\prime}$, $b^{\prime}$ and $\ell_{4}$ have additional decay channels to SM particles and $\phi$, provided they are kinematically allowed. In addition, as the usual setup in type-II 2HDM, we assign a $Z_{2}$ charge for each field to forbid tree level flavor-changing neutral currents and to simplify the scalar potential.

More explicitly, the quantum numbers of SM fermions and the two Higgs doublets $\Phi_{1}$ and $\Phi_{2}$ are listed in Table 1, and those of the fourth-generation fermions and the new scalar $\phi$ in Table 2. Gauge anomaly cancellation explicitly requires the relationship $9x_{q}+3x_{q4}=x_{\ell 4}$. In our numerical studies, we take $x_{q}=x_{q4}=1/3$ to be the same as their baryon number. Furthermore, the charge assignment of the SM singlet scalar field under $U(1)_{X}$ plays an important role to forbid the scalar $\phi$ to decay into SM particles and Higgs bosons, which makes $\phi$ a suitable DM candidate. The DM phenomenology will be discussed later. Also, we assume the $U(1)_{X}$ to be spontaneously broken at a certain energy scale higher than the electroweak scale. As a result, we have a new heavy gauge boson $Z^{\prime}$ that is crucial in the scattering cross section between DM and nucleon.

Unlike the sequential fourth generation, the SM gauge charges of the left- and right-handed fourth-generation leptons are interchanged. Such a setup can lead to an enhancement in the muon $g-2$ at the one-loop level due to the interactions between leptons and the signet scale field $\phi$, to be discussed in detail in the next section.

With the assignment $x_{\phi}=x_{\ell 4}$, the interaction Lagrangian for the SM leptons, the four-generation leptons and the scalar boson $\phi$ can be written down as

[TABLE]

where the flavor index $i=e,\mu,\tau$. We denote the mass of $e_{4}$ by $m_{e_{4}}$. Because of the $U(1)_{X}$ charges, such interactions are always lepton flavor-violating. For simplicity and concreteness, we will take the Yukawa couplings

[TABLE]

to be real in the following numerical studies. As far as the muon anomalous magnetic moment is concerned, only $f^{L}_{4\mu}$ and $f^{e}_{4\mu}$ are relevant.

Following the notations in Ref. Branco:2011iw , the two Higgs doublets $\Phi_{1}$ and $\Phi_{2}$ are parameterized as

[TABLE]

where $i=1,2$, $v_{1}=v\cos\beta$ and $v_{2}=v\sin\beta$ with $v=246$ GeV. The physical neutral scalars $h$ and $H$ can be expressed as linear combinations of $\rho_{1}$ and $\rho_{2}$:

[TABLE]

where $\alpha$ is a mixing angle among the neutral components. The scalar potential in the model includes two parts:

[TABLE]

where $V_{\text{2HDM}}$ is the scalar potential for the usual 2HDMs with a $Z_{2}$ symmetry softly broken by the $m_{12}^{2}$ terms and $V_{\phi}$ contains terms involving purely $\phi$ as well as mixing with $\Phi_{1,2}$. Here we assume that the new scalar field $\phi$ does not develop a vacuum expectation value (VEV). The singlet scalar mass $m_{\phi}$ and the coupling strength of the $h$-$\phi$-$\phi$ and $H$-$\phi$-$\phi$ vertices, denoted respectively by $\lambda_{h}$ and $\lambda_{H}$, can be derived from the above scalar potential to be

[TABLE]

We will consider the scenario where $\lambda_{H}\ll 1$ and, as a consequence, the coupling

[TABLE]

The Yukawa couplings of up-type fermions $y^{h}_{u}$ and down-type fermions $y^{h}_{d}$ are

[TABLE]

Note that the wrong-sign scenario can be realized when $\cos(\beta-\alpha)\lesssim 1/\tan\beta$ and $\tan\beta\gtrsim 2$ Das:2017mnu ; Han:2017etg . Unless otherwise specified, we adopt the following benchmark mass spectrum similar to the one given in Ref. Das:2017mnu for the numerical results presented in the following sections:

[TABLE]

III Muonic observables and constraints

In additional to the SM calculations, the muon anomalous magnetic dipole moment $a_{\mu}$ receives a major contribution from the one-loop diagram mediated by the fourth-generation charged lepton $e_{4}$ and the scalar boson $\phi$ under our setup, as shown in Fig. 1, Explicitly, we have Jegerlehner:2009ry

[TABLE]

The term involving $m_{e_{4}}$ gives the major contribution. Other diagrams, including those mediated by CP-even, CP-odd and charged Higgs bosons, give a result about four orders of magnitude smaller and are neglected. It is easy to see that $\Delta a_{\mu}$ depends upon three parameters of the model: $f_{4\mu}$, $m_{e_{4}}$ and $m_{\phi}$.

In Fig. 2, we show the parameter space in blue band that fits the observed $\Delta a_{\mu}$ at $1\sigma$ level. The grey region is excluded by the requirement of $\phi$ being the dark matter candidate and its decay to $e_{4}$ being kinematically forbidden. In Fig. 2, we fix the mass of $\phi$ to be $400$ GeV, and the coupling strength $f_{4\mu}$ is then found to fall between $0.04$ and $0.08$ for the heavy charged lepton up to about $1$ TeV. With $f_{4\mu}=0.05$ fixed in Fig. 2, we observe that the mass of $e_{4}$ is preferred to be in the window of $300$ GeV to $800$ GeV. As shown in the next section, the DM mass $m_{\phi}$ will be further restricted by the relic density measurement and direct detection bound.

In addition to offering a possible account of the muon $g-2$ anomaly, the interactions in Eq. (1) with the assumption in Eq. (2) also lead to lepton flavor-violating processes that should be controlled to be compatible with observed limits. Here we consider the following flavor-changing rare decays of muon and tau: $\mu^{-}\rightarrow e^{-}\gamma$, $\tau^{-}\rightarrow e^{-}\gamma$, $\tau^{-}\rightarrow\mu^{-}\gamma$ and $\mu^{-}\rightarrow e^{-}e^{-}e^{+}$. The first three processes occur through the one-loop diagrams analogous to Fig. 1 with the initial- and final-state muon replaced by the appropriate leptons, while the last one involves both penguin and box diagrams.

As an explicit example, the branching ratio of $\mu^{-}\rightarrow e^{-}\gamma$ is approximately given by Calibbi:2017uvl

[TABLE]

where the Fermi constant $G_{F}=1.166\times 10^{-5}~{}\text{GeV}^{-2}$ and

[TABLE]

The branching ratios of $\tau\to\mu\gamma$ ($\tau\to e\gamma$) can be obtained by replacing $m_{\mu}$ in Eq. (LABEL:eq:br) with $m_{\tau}$ and replacing $f_{4e}$ ($f_{4\mu}$) with $f_{4\tau}$ in Eq. (11). The analytical result of $\text{BR}(\mu\to eee)$ is more involved and given in the Appendix.

Currently, the upper bounds on the charged lepton flavor-violating decay modes are TheMEG:2016wtm ; Patrignani:2016xqp ; Pruna:2018egr :

[TABLE]

all quoted at the $90\%$ confidence level. With $f_{4i}$ assumed to be real, the bounds in Eq. (12) impose stringent constraints on the coupling products $f_{4e}f_{4\mu}$, $f_{4e}f_{4\tau}$ and $f_{4\tau}f_{4\mu}$. To satisfy the current limits, $f_{4e}$ and $f_{4\tau}$ are found respectively to be five and three orders of magnitude smaller than $f_{4\mu}\simeq 0.05$.

Because of the above-mentioned hierarchy among the $f_{4i}$ couplings, the box diagram for the $\mu\to eee$ decay is negligible as compared to the penguin diagram because it is proportional to $f_{4\mu}(f_{4e})^{3}$ while the latter involves just $f_{4\mu}f_{4e}$. Furthermore, the branching ratio of $\mu^{-}\rightarrow e^{-}e^{-}e^{+}$ has the quasi model-independent relation with $\mu^{-}\rightarrow e^{-}\gamma$ Kuno:1999jp ; Feldmann:2016hvo :

[TABLE]

Numerically, we have found the constraints from the radiative decays stronger than that from the three-body decay. The couplings $f_{4e}$ and $f_{4\tau}$ are found to be less contained in other charged lepton flavor-changing decays, $\tau\to 3\mu$, $\tau\to 3e$ and $\tau\to\mu ee$.

Besides, lepton flavor-violating decays of the $Z$ boson can be induced at the one-loop level due to the interactions in Eq. (1), as illustrated in Fig. 3. The branching ratio of $Z\rightarrow\ell_{i}^{\pm}\ell_{j}^{\mp}$ is given by Illana:2000ic

[TABLE]

where $\ell_{i,j}=e,\mu,\tau$ and the $Z$ boson total decay width $\Gamma_{Z}\simeq g^{2}/(4\pi\cos^{2}\theta_{W})m_{Z}$, $g$ is the $SU(2)_{L}$ gauge coupling constant, and $\theta_{W}$ is the weak mixing angle. The couplings in Eq. (LABEL:k) are given by

[TABLE]

where $g^{\prime}\equiv g/(\cos\theta_{W})$ and

[TABLE]

Currently, the experimental upper bounds Illana:2000ic

[TABLE]

constrain $f_{4e}f_{4\mu}$, $f_{4e}f_{4\tau}$ and $f_{4\mu}f_{4\tau}$, respectively. To explain the muon $g-2$ anomaly, the preferred value of $f_{4\mu}$ is around $0.05$ when masses of $\phi$ and $e_{4}$ are of ${\cal O}(100)$ GeV as seen in Fig. 2. Given this parameter space preferred by the observed $\Delta a_{\mu}$, the upper bounds on $f_{4e}$ and $f_{4\tau}$ extracted from Eq. (17) turn out to be much less stringent than the bounds obtained from the rare charged lepton decays in Eq. (12).

IV Dark Matter

In this section, we study the phenomenology of DM candidate $\phi$, including its relic density and constraint from the null result of direct search.

The present dark matter relic density is determined by its cross section of annihilation to SM particles. Two $\phi^{\prime}s$ can annihilate into several possible final states of SM particles in pairs, including leptons, quarks, gauge bosons and Higgs bosons. The first case involves a fourth-generation lepton as the mediator in the $t$-channel process. However, this channel gives a tiny contribution when we take into account the rare decay limits of charged leptons discussed previously. Therefore, the annihilation cross section is dominated by diagrams mediated by the 125-GeV Higgs boson $h$ and the $Z^{\prime}$ boson, between which the former plays a dominant role.

Fig. 4 shows the parameter space in the $m_{\phi}$-$\lambda_{h}$ plane that renders the observed DM relic density $\Omega h^{2}=0.1200\pm 0.0012$ Aghanim:2018eyx , where $\lambda_{h}$ parameterizes the coupling strength of the $h$-$\phi$-$\phi$ vertex which reads as $2\lambda_{h}m_{W}/g$. The black, red and blue bands are respectively the results with $\tan\beta=3$, $5$ and $10$, and the angle $\alpha$ is chosen to comply with the wrong-sign scenario. The colored bands show a few structures due to various kinematical reasons. As the DM mass is close to half of the Higgs boson mass, the sharp drop reflects the resonance enhancement in the annihilation cross section. Similarly, there are then a few more drops as the DM mass crosses the thresholds to annihilate into a pair of $W$ bosons, $Z$ bosons, Higgs bosons and top quarks, respectively. In this plot, we also draw a grey area that is excluded by the upper limit of Higgs boson invisible decay branching ratio $\text{BR}(h\to\phi\phi)\lesssim 0.24$ Khachatryan:2016whc .

We now turn to the spin-independent elastic scattering between the DM and a nucleon, related to direct search experiments. The scattering process receives contributions that features the exchanges of Higgs and $Z^{\prime}$ bosons. In the low-energy approximation, the DM-nucleon (proton or neutron) cross section can be expressed as

[TABLE]

with negligible interference between the two diagrams due to the large mass separation between $h$ and $Z^{\prime}$. The contribution from the Higgs-mediated diagram reads

[TABLE]

where $m_{p}$ and $m_{n}$ are the masses of proton and neutron, respectively, and

[TABLE]

In the above equation, $f^{p(n)}_{Tq}$ represent the contributions of light quarks to the mass of the proton (neutron) and $\alpha^{S}_{q}\approx-i\lambda_{h}m_{q}/m^{2}_{h}$ is the effective coupling after integrating out the Higgs boson in the low-energy approximation. The second term represents the interaction of DM with the gluon scalar density in the nucleon, with $f^{p(n)}_{TG}=1-\sum_{q=u,d,s}f^{p(n)}_{Tq}$. Values of these form factors used in the numerical analysis are taken as Junnarkar:2013ac ; Hoferichter:2015dsa ; Alarcon:2011zs ; Alarcon:2012nr ; Cheng:2012qr

[TABLE]

The other contribution to the spin-independent DM-nucleon scattering comes from the $Z^{\prime}$-mediated diagram. This cross section is given by

[TABLE]

where $e^{\prime}$ denotes the gauge coupling strength of the $U(1)_{X}$ group, and $x_{q}$ and $x_{\phi}$ are the $U(1)_{X}$ charges of SM quarks and dark matter, respectively.

The blue curve in Fig. 5 shows the upper limit of elastic scattering cross section that is consistent with the null result of XENON1T experiment Aprile:2017iyp if only the Higgs-mediated diagram is considered. The black band and the grey region are quoted from the results with correct dark matter relic density with $\tan\beta=3$ and $\cos(\beta-\alpha)=0.525$ and exclusion by Higgs invisible decay as given in Fig. 4. Most of the region with $m_{\phi}$ heavier than $m_{h}/2$ are allowed, except for a small mass window around $m_{\phi}\simeq 80$ GeV.

Other curves in yellow, green and red represent the constraints for different $U(1)_{X}$ gauge coupling strength $e^{\prime}=0.013,~{}0.015$ and $0.018$, respectively. Similarly, the region above each curve is excluded. Clearly, the elastic cross section is very sensitive to the $U(1)_{X}$ gauge coupling strength, and the $Z^{\prime}$-mediated diagram can easily dominate over the Higgs boson contribution when the new gauge coupling is turned on. For example, $m_{\phi}\lesssim 300$ GeV is excluded in the case of $e^{\prime}=0.018$.

Finally, we make a brief comment on the phenomenology of $Z^{\prime}$ in the model at hadron colliders. Since the $Z^{\prime}$ boson couples to quarks, it can be searched through the dijet resonance. However, the usual resonance search in dilepton channel would not apply since the SM leptons are $U(1)_{X}$ charge neutral. And we have checked that the numerical results presented in this work satisfy the $Z^{\prime}$ lower bound set by the LHC dijet resonance search Lee:2011jk . Furthermore, since $Z^{\prime}$ couples to the fourth-generation leptons and dark matter as well, it mainly decays into dark matter particles if $m_{Z^{\prime}}\lesssim 2m_{\ell_{4}}$. When the decay channels to fourth-generation leptons are kinematically allowed, the branching ratios to $\ell_{4}$ become dominant, as we can see in Fig. 6.

Therefore, the dijet branching ratio will be suppressed. The signatures at the LHC would be mono-jet with large missing energy for a light $Z^{\prime}$. For a heavy $Z^{\prime}$ that decays dominantly into two fourth-generation charged leptons, one should focus on the signature of two charged leptons with missing energy.

V Conclusions

The measurement of muon $g-2$, with a $\sim 3\sigma$ deviation away from the present SM predictions, may present hints of new physics. Furthermore, evidence showing the existence of dark matter certainly call for an extension of SM. In this paper, we have considered a type-II two-Higgs doublet model with an additional generation of fermions, a SM singlet scalar and a new $U(1)_{X}$ gauge group. With non-canonical $SU(2)_{L}$ charge assignments to the fourth-generation leptons, we showed that the additional contributions of fourth-generation charged leptons with the singlet scalar in the loop could fix the tension between muon $g-2$ measurement and theory predictions. In addition, flavor-changing rare decays of charged leptons and $Z$ boson could be induced in this model. And we have checked that the parameter space to explain muon $g-2$ anomaly is consistent with these constraints.

With the assumed $U(1)_{X}$ charges of fields in this model, the singlet scalar was found to be a suitable dark matter candidate that fits the observed relic density today. However, the LHC data of Higgs invisible decay excludes the possibility of dark matter being lighter than $m_{h}/2$. Furthermore, the scattering cross section between dark matter and nucleon is sensitive to the gauge coupling strength of $U(1)_{X}$. As a result, the null results in the direct search of dark matter experiments impose stringent constraints on dark matter mass even when the $U(1)_{X}$ gauge coupling is small.

Acknowledgments

C. W. C. would like to thank the hospitality of the New High Energy Theory Center at Rutgers University where part of this work was done. This research was supported in part by the Ministry of Science and Technology of Taiwan under Grant No. MOST 105-2112-M-003-010-MY3 (C. R. C.) and MOST 104-2628-M-002-014-MY4 (C. W. C.).

Appendix A Details of $\mu\to 3e$

There are two kinds of Feynman diagrams that contribute to $\mu\to 3e$ decay at one-loop level: penguin diagram and box diagram, as shown in Fig. 7.

Following the notations in Refs. Kuno:1999jp ; Feldmann:2016hvo , the effective Lagrangian can be described as

[TABLE]

where

[TABLE]

Note that, with this definition, the branching ratio of $\mu\to e\gamma$ reads as

[TABLE]

Therefore, the branching ratio of $\mu\to 3e$ can be expressed as

[TABLE]

In our case, we obtain

[TABLE]

where

[TABLE]

with $x_{\phi}\equiv m_{\phi}^{2}/m_{e_{4}}^{2}$.