Lepton Flavor Violation and Collider Searches in a Type I + II Seesaw Model

TL;DR

This paper investigates a combined type I + II seesaw model for neutrino masses, analyzing lepton flavor violation and collider signals, highlighting the complementary nature of muon decay experiments and collider searches in probing new physics.

Contribution

It introduces a model where type I and II seesaw mechanisms arise from spontaneous symmetry breaking of an extended gauge group, and studies its implications for lepton flavor violation and collider phenomenology.

Findings

Lepton flavor violation constraints depend on neutrino mass ordering.

Collider searches for doubly charged scalars are complementary to muon decay experiments.

The combined approach can identify the underlying seesaw mechanism.

Abstract

Neutrino are massless in the Standard Model. The most popular mechanism to generate neutrino masses are the type I and type II seesaw, where right-handed neutrinos and a scalar triplet are augmented to the Standard Model, respectively. In this work, we discuss a model where a type I + II seesaw mechanism naturally arises via spontaneous symmetry breaking of an enlarged gauge group. Lepton flavor violation is a common feature in such setup and for this reason, we compute the model contribution to the $\mu \rightarrow e\gamma$ and $\mu \rightarrow 3e$ decays. Moreover, we explore the connection between the neutrino mass ordering and lepton flavor violation in perspective with the LHC, HL-LHC and HE-LHC sensitivities to the doubly charged scalar stemming from the Higgs triplet. Our results explicitly show the importance of searching for signs of lepton flavor violation in collider and muon decays. The conclusion about which probe yields stronger bounds depends strongly on the mass ordering adopted, the absolute neutrino masses and which much decay one considers. In the 1-5 TeV mass region of the doubly charged scalar, lepton flavor violation experiments and colliders offer orthogonal and complementary probes. Thus if a signal is observed in one of the two new physics searches, the other will be able to assess whether it stems from a seesaw framework.

Tables

Table 1. Table 1 : Table with the best-fit parameters that enter in the neutrino mass mixing according to [ 115 ] .

(4.11)

Table 2. Table 2 : Table with the theoretical predictions for the μ → e ​ γ → 𝜇 𝑒 𝛾 \mu\rightarrow e\gamma and μ → 3 ​ e → 𝜇 3 𝑒 \mu\rightarrow 3e decay for the type I + I ​ I 𝐼 𝐼 𝐼 I+II seesaw setup encompassing normal (NO) and inverted (IO) mass ordering. We highlight that the mass of the doubly charged scalar is in G ​ e ​ V 𝐺 𝑒 𝑉 GeV units. The neutrino masses in the IO scenario are given by m ν ​ 2 2 = m ν ​ 1 2 − Δ ​ m 21 2 superscript subscript 𝑚 𝜈 2 2 superscript subscript 𝑚 𝜈 1 2 Δ superscript subscript 𝑚 21 2 m_{\nu 2}^{2}=m_{\nu 1}^{2}-\Delta m_{21}^{2} , m ν ​ 3 2 = m ν ​ 2 2 − Δ ​ m 23 2 superscript subscript 𝑚 𝜈 3 2 superscript subscript 𝑚 𝜈 2 2 Δ superscript subscript 𝑚 23 2 m_{\nu 3}^{2}=m_{\nu 2}^{2}-\Delta m_{23}^{2} , whereas in NO case are m ν ​ 2 2 = m ν ​ 1 2 − Δ ​ m 21 2 superscript subscript 𝑚 𝜈 2 2 superscript subscript 𝑚 𝜈 1 2 Δ superscript subscript 𝑚 21 2 m_{\nu 2}^{2}=m_{\nu 1}^{2}-\Delta m_{21}^{2} , m ν ​ 3 2 = m ν ​ 1 2 + Δ ​ m 31 2 superscript subscript 𝑚 𝜈 3 2 superscript subscript 𝑚 𝜈 1 2 Δ superscript subscript 𝑚 31 2 m_{\nu 3}^{2}=m_{\nu 1}^{2}+\Delta m_{31}^{2} .

Benchmark	Hierarchy	BR ($\mu \to e\gamma$)	$\mathrm{BR}\left(\mu \to 3e\right)$
$m_{\nu 1}=0.1$ eV, $v_{s11}=1$ eV	IO	$0.024/m_{S^{\pm \pm }}^4$	$7.5/m_{S^{\pm \pm }}^4$
$m_{\nu 1}=0.1$ eV, $v_{s11}=100$ eV	IO	$2.4\times {10}^{-10}/m_{S^{\pm \pm }}^4$	$7.5\times {10}^{-8}/m_{S^{\pm \pm }}^4$
$m_{\nu 1}=0.01$ eV, $v_{s11}=1$ eV	IO	$0.08/m_{S^{\pm \pm }}^4$	$3.5/m_{S^{\pm \pm }}^4$
$m_{\nu 1}=0.01$ eV, $v_{s11}=100$ eV	IO	$8\times {10}^{-10}/m_{S^{\pm \pm }}^4$	$3.5\times {10}^{-8}/m_{S^{\pm \pm }}^4$
$m_{\nu 1}=0.1$ eV, $v_{s11}=1$ eV	NO	$0.44/m_{S^{\pm \pm }}^4$	$106/m_{S^{\pm \pm }}^4$
$m_{\nu 1}=0.1$ eV, $v_{s11}=100$ eV	NO	$4.4\times {10}^{-9}/m_{S^{\pm \pm }}^4$	$1.1\times {10}^{-6}/m_{S^{\pm \pm }}^4$
$m_{\nu 1}=0.01$ eV, $v_{s11}=1$ eV	NO	$0.06/m_{S^{\pm \pm }}^4$	$2.5/m_{S^{\pm \pm }}^4$
$m_{\nu 1}=0.01$ eV, $v_{s11}=100$ eV	NO	$6.2\times {10}^{-10}/m_{S^{\pm \pm }}^4$	$2.5\times {10}^{-8}/m_{S^{\pm \pm }}^4$

Equations

$$\psi_{aL}=\left(\begin{array}[]{c}\nu_{a}\\ e_{a}\\ \nu_{a}^{c}\end{array}\right)_{L}\sim\left(\mathbf{1},\mathbf{3},-1/3\right),\text{ \ }e_{aR}\sim(\mathbf{1},\mathbf{1},-1),$$

$$Q_{\alpha L}$$

$$u_{\alpha R}$$

$$Q_{3L}$$

$$u_{3R}$$

$$\chi=\left(\begin{array}[]{c}\chi_{1}^{0}\\ \chi_{2}^{-}\\ \chi_{3}^{0}\end{array}\right)\sim\left(\mathbf{1},\mathbf{3},-1/3\right),\text{ }\eta=\left(\begin{array}[]{c}\eta_{1}^{0}\\ \eta_{2}^{-}\\ \eta_{3}^{0}\end{array}\right)\sim\left(\mathbf{1},\mathbf{3},-1/3\right),\text{ }\rho=\left(\begin{array}[]{c}\rho_{1}^{+}\\ \rho_{2}^{0}\\ \rho_{3}^{+}\end{array}\right)\sim\left(\mathbf{1},\mathbf{3},2/3\right).$$

$$\left\langle\chi\right\rangle=\frac{1}{\sqrt{2}}\left(\begin{array}[]{c}0\\ 0\\ v_{\chi}\end{array}\right),\text{ }\left\langle\eta\right\rangle=\frac{1}{\sqrt{2}}\left(\begin{array}[]{c}v_{\eta}\\ 0\\ 0\end{array}\right),\text{ }\left\langle\rho\right\rangle=\frac{1}{\sqrt{2}}\left(\begin{array}[]{c}0\\ v_{\rho}\\ 0\end{array}\right).$$

$${\mathcal{L}}_Y^l=h_{ab}^l{\overline{\psi }}_{aL}\rho e_{bR}+h.c..$$

$$m_l=h_{ab}^l\frac{v_{\rho }}{\sqrt{2}}.$$

$${\mathcal{L}}_{LFC}$$

$${\mathcal{L}}_{LFV}$$

$$m_{u,c}=\frac{-h_{11,22}^u{v}_{\rho }}{\sqrt{2}},m_t=\frac{h_{33}^u{v}_{\eta }}{\sqrt{2}},$$

$$m_{d,s}=\frac{h_{11,22}^d{v}_{\eta }}{\sqrt{2}},m_b=\frac{h_{33}^d{v}_{\rho }}{\sqrt{2}}.$$

$$m_U=\frac{h^U{v}_{\chi }}{\sqrt{2}},m_D=\frac{h_{11,22}^D{v}_{\chi }}{\sqrt{2}}$$

$$S=\left(\begin{tabular}[]{lll}$S_{11}^{0}$&$S_{12}^{-}$&$S_{13}^{0}$\\ $S_{12}^{-}$&$S_{22}^{--}$&$S_{23}^{-}$\\ $S_{13}^{0}$&$S_{23}^{-}$&$S_{33}^{0}$\end{tabular}\right)\sim\left(\mathbf{1},\mathbf{6},-2/3\right).$$

$${\mathcal{L}}_Y^l=f_{ab}^{\nu }({\overline{\psi }}_{aL}{)}_m({\psi }_{bL}^c{)}_n{S}_{mn}+h.c..$$

$$\left\langle S\right\rangle=\frac{1}{\sqrt{2}}\left(\begin{tabular}[]{lll}$v_{s_{11}}$&$0$&$v_{s_{13}}$\\ $0$&$0$&$0$\\ $v_{s_{13}}$&$0$&$\Lambda$\end{tabular}\right)$$

$$\mathcal{L}_{mass}^{\nu}=-\frac{1}{2}\left(\begin{array}[]{cc}\overline{\nu}_{aL}&\overline{\nu}_{aR}^{c}\end{array}\right)M_{\nu}\left(\begin{array}[]{c}\nu_{bL}^{c}\\ \nu_{bR}\end{array}\right)+h.c.$$

$$M_{\nu}=\left(\begin{array}[]{cc}M_{L}&M_{D}\\ M_{D}^{T}&M_{R}\end{array}\right)$$

$$M_L$$

$$M_D$$

$$M_R$$

$$(m_{\nu }{)}_{ab}=\sqrt{2}\left(v_{s_{11}}-\frac{v_{s_{13}}^2}{\Lambda }\right)(f^{\nu }{)}_{ab},$$

$$(M_{{\nu }_R}{)}_{ab}=\sqrt{2}\Lambda (f^{\nu }{)}_{ab}.$$

$${\mathcal{D}}_{\mu }\phi =\left[{\partial }_{\mu }-i{g}_L{W}_{\mu }^m\frac{{\lambda }^m}{2}-i{g}_N{N}_{\phi }{W}_{\mu }^N\right]\phi ,$$

$$\frac{g_{L}}{2}W_{\mu}^{m}\lambda^{m}=\frac{g_{L}}{2}\left(\begin{array}[]{ccc}\left(W_{\mu}^{3}+\frac{1}{\sqrt{3}}W_{\mu}^{8}\right)&\sqrt{2}W_{\mu}^{+}&\sqrt{2}U_{\mu}^{0}\\ \sqrt{2}W_{\mu}^{-}&\frac{g_{L}}{2}\left(-W_{\mu}^{3}+\frac{1}{\sqrt{3}}W_{\mu}^{8}\right)&\sqrt{2}W_{\prime\mu}^{-}\\ \sqrt{2}U_{\mu}^{0\dagger}&\sqrt{2}W_{\prime\mu}^{+}&-\frac{2}{\sqrt{3}}W_{\mu}^{8}\end{array}\right),$$

$$v_{s_1}\le 2GeV,$$

$$m_{Z^{\prime }}\simeq 0.4v_{\chi },m_{W^{\prime }}\simeq 0.32v_{\chi }$$

$$m_{Z^{\prime }}>4\mathrm{TeV}(\mathcal{L}=36f{b}^{-1}),m_{Z^{\prime }}>6.4\mathrm{TeV}(\mathcal{L}=3a{b}^{-1}).$$

$$\mathcal{L}=f_{ab}^{\nu }{\bar{L}}_a^c{\Delta }^{\ast }{L}_b$$

$$\Delta=\left(\begin{array}[]{cc}S^{0}&S^{-}\\ S^{-}&S^{--}\\ \end{array}\right),$$

$$\Gamma (S^{\pm \pm }\to l_a^{\pm }{l}_b^{\pm })=1/(2\pi x)|f_{ab}^{\nu }{|}^2{m}_{S^{\pm \pm }}^2,$$

Full text

1 Introduction

The observation of neutrino oscillations implies in non-zero neutrino masses. Their masses are much smaller than any other in the Standard Model (SM) spectrum. Several mechanism have surfaced trying to explain the smallness of the neutrino masses [1, 2, 3, 4, 5]. In the type I seesaw, the existence of heavy right-handed neutrinos is evoked, whereas in the type II seesaw a scalar triplet is added to the SM. This scalar triplet couples to the SM lepton doublets and features a neutral scalar that develops a small vacuum expectation value giving rise to tiny neutrino masses [6, 7, 8, 9, 10]. This mechanism can elegantly explain the masses of the three active neutrinos in the SM and lead to several phenomenological imprints in collider, low energy observables and leptogenesis [11, 12, 12, 13, 14, 15, 16, 17].

On the other side, we do not know why we have three generations of fermions in the SM. Theoretically speaking, it would nice to have a model where these two problems are simultaneously addressed. Several models have been proposed where the number of generations is addressed by anomaly cancellation and asymptotic freedom arguments, and they are known as 3-3-1 models [18, 19, 20]. These models have an enlarged gauge sector, $SU(3)_{c}\times SU(3)_{L}\times U(1)_{N}$  (3-3-1 for short). Notice that $SU(2)_{L}\times U(1)_{Y}$ in the SM gives place to $SU(3)_{L}\times U(1)_{N}$. Therefore, the fermions will now be arranged in the fundamental representation of $SU(3)_{L}$, i.e. triplets. The same happens for scalars fields, also sorted in triplets in order to generate fermion masses. Several versions based on the 3-3-1 symmetry have been proposed in the literature where the replication of fermion generations are explained [21, 22, 23, 24, 25].

Not all of these realizations are capable of explaining neutrino masses at the same time, though. In this work, we focus on a model which can account for neutrino masses, known as 3-3-1 model with right-handed neutrinos [20, 26, 27, 28]. Originally the model has only three scalar triplets. With scalar triplets one can generate two mass degenerate neutrinos and a massless one [29], which is in conflict with existing data [30]. The use of high dimensional effective operators has been put forth in the attempt to break the degeneracy [31, 32]. In this setup, the smallness of the neutrino masses and the oscillation pattern is not successfully explained. The most simple way to nicely solve this issue is by adding a scalar sextet [33, 34]. The interesting aspect of this scalar sextet is that after spontaneous symmetry breaking it breaks down to a scalar triplet, two doublet scalars and a scalar singlet field. The scalar triplet is exactly the one desired to perform the type II seesaw mechanism. Therefore, in this way, the type II seesaw arises as a result of the spontaneous symmetry breaking. Several phenomenological aspects of the 3-3-1 with right-handed neutrinos have been explored in the past [35, 36, 37, 38, 39, 40], but our work differs from those because,

• •

We explicitly compute the $\mu\rightarrow e\gamma$ and $\mu\rightarrow 3e$ decays;

• •

We explore their connection to neutrino mass ordering;

• •

We put our results into perspective with collider and lepton flavor violation bounds.

This work is structured as follows: In section II we briefly describe the model and explain how neutrino masses are incorporated, in section III we present the relevant collider bounds, in section IV we derive the lepton flavor violating muon decays and then draw our conclusions. In Appendix, we provide a details discussion of our reasoning and a general derivation of the $\mu\rightarrow e\gamma$ decay.

2 Model with right-handed neutrinos and scalar sextet

There are several models based on the 3-3-1 gauge group. The model discussed here differs from the original proposal [18, 19, 20] because of the presence of three right-handed neutrinos. This model does not suffer from a Landau pole at the TeV scale as the previous version. For this reason, it has attracted lots of attention in the past decade [41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52]. In this section, we will briefly discuss the model to ease our reasoning. We start with the fermion content.

2.1 Fermion Content

In this model, the left-handed leptons are arranged in the fundamental representation of $SU(3)_{L}$, whereas right-handed leptons as a singlet, as follows,

[TABLE]

where $a=1,2,3$ runs through the fermion generations. The numbers in parenthesis represent the quantum number of the fields under $SU(3)_{C}$, $SU(3)_{L}$ and $U(1)_{N}$, respectively. For instance, the $\psi_{aL}$ is a singlet under $SU(3)_{C}$, triplet under $SU(3)_{L}$ and has a charge equal $-1/3$ under $U(1)_{N}$. Notice that the third component of the leptonic triplet is a right-handed neutrino since $\left(\nu_{a}^{c}\right)_{L}=\nu_{R}^{c}$, explaining why this model is known as 3-3-1 model with right-handed neutrinos.

In a similar vein, the quarks are arranged as follows,

[TABLE]

[TABLE]

where $\alpha=1,2$. We highlight that the first two generations of quarks are placed in the anti-triplet representation of $SU(3)_{L}$. This has to be the case in order to cancel gauge anomalies. The quarks $D_{1,2}$ are down-type heavy quarks, in the sense, they have the same hypercharge as the SM quark down. A similar logic applies to the heavy up-quark $U$. These quarks have masses proportional to the scale of symmetry breaking of the 3-3-1 symmetry which should lie at several TeV to be consistent with the non-observation of such exotic quarks [53, 54]. The scale of symmetry breaking of the 3-3-1 symmetry will be kept sufficiently high to be consistent with this result. We will address the fermion masses below.

2.2 Fermion Masses

To generate fermion masses we need to invoke three scalar triplets of the following form,

[TABLE]

These scalar triplets are sufficient to successfully yield masses to all fermions, except neutrinos. The spontaneous symmetry breaking goes as: first SU(3)_{L}\otimes U(1)_{X}\overset{\left\langle\chi\right\rangle}{\rightarrow}SU(2)_{L}\otimes U(1)_{Y},\while$\ \ SU(2)_{L}\otimes U(1)_{Y}\overset{\left\langle\eta\right\rangle,\left\langle\rho\right\rangle}{\rightarrow}U(1)_{Q}$. In other words, when $\chi$ develops vacuum expectation values the 3-3-1 symmetry is broken reproducing the SM gauge group, which then breaks into quantum electrodynamics after $\eta$ and $\rho$ acquire a vacuum expectation value. For the purpose of this work, it easier to work on the broken phase where,

[TABLE]

For simplicity we will adopt $v_{\eta}=v_{\rho}$. This assumption is typically made to simplify the diagonalization of the mass matrices involving the scalars [43]. Moreover, we take $v_{\eta}^{2}+v_{\rho}^{2}\sim 246^{2}{\rm GeV^{2}}$, this ought to be enforced to generate the correct masses for the W and Z bosons in the SM, as occurs in models with extra scalars contributing to gauge boson masses [55, 56].

We have explained in detail how each mass is generated in the Appendix, for this reason, in what follows we will just briefly consider each sector separately.

2.2.1 Charged Leptons Masses

The masses for the charged leptons are generated via the lagrangian,

[TABLE]

Notice that this term conserves lepton number. One possible term that one could write down is $\epsilon_{ijk}\,\overline{\psi^{c}}_{L\,i}\psi_{Lj}\rho_{k}$, which does not conserve lepton flavor. The reason why we did not include this term in Eq.(2.14) is that there is a set of discrete symmetries that will invoke later on to prevent mixing between the SM quarks and the exotic ones. One of them requires $\rho\rightarrow-\rho$, which forbids the term above. However, we need to impose $e_{R}\rightarrow-e_{R}$ to engender masses for charged leptons. When the field $\rho_{2}^{0}$ develops a vacuum expectation value, $v_{\rho}$, we find,

[TABLE]

2.2.2 Quarks Masses

The quark masses could stem from two sources, one where flavor is conserved (LFC) and other where it is violated (LFV) as follows,

[TABLE]

and

[TABLE]

Nevertheless, the LFV lagrangian is problematic because it induces mixing between the SM quarks and the exotic ones. This mixing could alter the properties of the SM quarks. Thus we need to eliminate them. To do so, we invoke a set of $Z_{2}$ symmetries where some fields are odd under: $u_{aR}\rightarrow-u_{aR}$, $d_{aR}\rightarrow-d_{aR}$, $D_{aR}\rightarrow-D_{aR}$, $\eta\rightarrow-\eta$, $\rho\rightarrow-\rho$. The reaming fields transform trivially. We emphasize that these discrete symmetries do not affect the other sectors of the model that is still capable of reproducing the SM features at low energy scales with no prejudice.

In summary, with these discrete symmetries the SM quarks get masses through Eq.(2.16) yielding,

[TABLE]

and,

[TABLE]

It is visible that their masses are proportional to $v_{\eta}$ and $v_{\rho}$ which are set at the electroweak scale to avoid flavor changing interactions and return the correct quark masses [57, 45, 58, 59, 60, 61, 62, 63].

Moreover, from Eq.(2.16) we find the exotic quarks masses,

[TABLE]

which are proportional to the scale of the 3-3-1 symmetry breaking, $v_{\chi}$, assumed to be sufficiently high to be in agreement with null results from collider searches for exotic quarks [64, 65, 54]. We will now turn our attention to neutrino masses.

2.2.3 Neutrino Masses- Type $I+II$ Seesaw

As far as neutrino masses are concerned, the scalar triplets do not suffice to successfully generate neutrino masses. Using Lie algebra one may notice that $\bar{\psi_{L}^{c}}\psi_{L}\sim(\mathbf{3}^{\ast}+\mathbf{6},-2/3)$. Thus, one may generate neutrino masses via the scalar triplet $\rho$ and a scalar sextet [20, 66, 67] with,

[TABLE]

However, the discrete symmetries mentioned previously prohibit the Yukawa term proportional to $\rho$ and for this reason, only the scalar sextet contributes leading to,

[TABLE]

Neutrino masses arise after the neutral scalars develop vacuum expectation value as follows,

[TABLE]

Notice that $v_{s11},v_{s13}$, and $\Lambda$ are the vacuum expectation value of the neutral scalars in the sextet.

With this vacuum structure we generate a mass matrix of the form,

[TABLE]

where $M_{\nu}$,

[TABLE]

with,

[TABLE]

We remind the reader that $M_{\nu}$ is a $6\times 6$ matrix and $M_{L},M_{D}$ and $M_{R}$ as $3\times 3$ matrices. After the diagonalization procedure we get for the active neutrinos,

[TABLE]

and for the right-handed ones,

[TABLE]

It is good timing to highlight a few things:

(i) the mass ratio $m_{\nu}/m_{\nu_{R}}$ is independent of the Yukawa coupling;

(ii) a scalar sextet breaks down to a triplet plus doublet plus singlet scalar, i.e. $\mathbf{6}\rightarrow\mathbf{3}+\mathbf{2}+\mathbf{1}$. In order words, the scalar sextet generates the triplet scalar used in the type II seesaw, the doublet scalar that induces Dirac masses, $M_{D}$, and a singlet scalar that yields right-handed Majorana masses, $M_{R}$. Therefore, the scalar sextet naturally gives rise to a type I +II seesaw mechanism via spontaneous symmetry breaking;

(iii) Setting $v_{s13}^{2}/\Lambda\sim v_{s11}$ the smallness of the active neutrino masses are justified by taking $v_{s11}$ to be around $1$ eV.

There are different possibilities to successfully generate neutrino masses. Since the vacuum expectation values of the fields in the scalar sextet are in principle arbitrary one can play with them and find different Yukawa couplings leading to the same neutrino masses. Be that as it may, one can draw important and insightful conclusions by adopting some simplifications. We will assume throughout that $v_{s13}^{2}/\Lambda\sim v_{s11}$ and $v_{s13}=v_{\rho}=v_{\eta}$. Therefore, if we choose $v_{s11}=1$ eV this automatically translates into $\Lambda\sim 10^{13}$ GeV, and $v_{s11}=100$ eV yields $\Lambda\sim 10^{11}$ GeV. Later on, we will present several benchmark scenarios taking $v_{s11}=1$ eV and $v_{s11}=100$ eV to investigate the impact of the neutrino mass ordering on the lepton flavor violating muon decays. With this information, one can now have an estimate of the right-handed neutrino masses since they are proportional to $\Lambda$.

For simplicity we will adopt that $v_{s13}^{2}/\Lambda\sim v_{s11}$ so that $(m_{\nu})_{ab}=\sqrt{2}f^{\nu}_{ab}$. In this way, we have a dominant type II seesaw mechanism with the neutrino masses governed by the vacuum expectation value of $S^{0}_{11}$. One can think of it as the vacuum expectation value of the triplet under $SU(2)_{L}$ in the broken phase in the usual type II seesaw study.

Looking at Eq.(2.21) we notice that $S^{0}_{33},S^{0}_{11},S^{-}_{12},S^{--}_{22}$ carry two units of lepton number and for this reason are called bileptons. The lepton flavor (number) is violated after the neutral components $S^{0}_{11}$ and $S^{0}_{33}$ acquire a vacuum expectation value. The singly charged scalar in the scalar sextet will not mix with other singly charged scalars that do not carry lepton number. The presence of flavor violation will be explored in this work in the context of muon decays.

2.3 Gauge Sector

We turn our attention to the gauge sector. The main point is to show that there are new gauge bosons with masses are proportional to the 3-3-1 scale of symmetry breaking and they are subject to stringent collider bounds. Showing this in a pedagogical manner requires us to start with the covariant derivative of $SU(3)_{L}\otimes U(1)_{N}$,

[TABLE]

where $g_{L}$ is the coupling constant of $SU_{L}\left(3\right)$ group, $g_{N}$ is the coupling constant of $U(1)_{N},$ $\lambda^{m}$ are the Gell-Mann Matrices with $m=1,...,8,$ $W_{\mu}^{m}$ are the gauge bosons in the adjoint representation of $SU(3)_{L}$, $W_{\mu}^{N}$ is the gauge field associated to $U(1)_{N}$ and $N_{\varphi}$ is the hypercharge associated to $U(1)_{N}$. Writing down the term proportional to $\lambda^{m}$ one finds,

[TABLE]

where $W_{\mu}^{\pm}=\left(W_{\mu}^{1}\mp iW_{\mu}^{2}\right)/\sqrt{2},$ $U_{\mu}^{0}=\left(W_{\mu}^{4}+iW_{\mu}^{5}\right)/\sqrt{2},U_{\mu}^{0\dagger}=\left(W_{\mu}^{4}-iW_{\mu}^{5}\right)/\sqrt{2},$ $W_{\mu}^{\prime\pm}=\left(W_{\mu}^{6}\pm iW_{\mu}^{7}\right)/\sqrt{2}$.

Therefore, the 3-3-1 model with right-handed neutrinos predicts the existence of new charged gauge bosons, $W^{\prime\pm}$, which are subject to intense searches at the LHC [68, 69], and exotic neutral gauge bosons $U^{0}$ and $U^{0\dagger}$ [70]. Moreover, from a combination of the $W_{8}$, $W_{3}$ and $W_{N}$ fields, we extract the SM photon and $Z$ bosons, as well as a massive $Z^{\prime}$ field. It is clear the model add five new gauge bosons to the SM, as a direct result of the extended gauge sector which predicted $N^{2}-1$ bosons, where $N=3$ in this case.

The masses of SM gauge bosons are slightly altered by the presence of the scalar sextet. This change is proportional to the vacuum expectation value $v_{s11}$ which is meant to be small. A similar conclusion is found in the usual type II seesaw mechanism. The bound that rises from the $\rho$ parameter enforces [55] at $3\sigma$,

[TABLE]

The masses of the new gauge bosons are all proportional to the scale of symmetry breaking of the model. For a complete spectrum and equations of the gauge boson masses, we refer to [43]. Although, some relations are quite useful to ease our reasoning. One can find that,

[TABLE]

i.e. $m_{Z^{\prime}}\simeq 1.25\,m_{W^{\prime}}$.

Hence, a lower mass bound on the $Z^{\prime}$ boson represents a direct lower mass bound on the $W^{\prime}$, similarly to what occurs in the minimal left-right model [68, 69]. One may wonder about the existence of important limits related collider searches for the $U^{0}$ gauge boson. Although, its mass is identical to the $W^{\prime}$ which is subject to much more intensive searches. At the end of the day, the most important gauge boson as far as collider searches are concerned will be the $Z^{\prime}$ field, as we explain further.

3 Collider Bounds

3.1 Searches for $Z^{\prime}$ bosons

Collider searches for heavy $Z^{\prime}$ bosons are quite popular because they typically feature a clear signal. If they couple to fermions and have a narrow width, they give rise to pronounced bumps in the dilepton or dijet invariant mass [71]. If the couplings to leptons are not very suppressed, the use of dilepton data is more promising because it is subject to a smaller SM background. In this model, the $Z^{\prime}$ couplings to leptons are not small. Using LHC data at $13$ TeV center-of-mass energy with $3.2$ $fb^{-1}$ of integrated luminosity the authors in [72] placed a lower mass bound of $3$ TeV. Later, in [73] this limit was improved using $36$ $fb^{-1}$ and $3$ $ab^{-1}$ of integrated luminosity finding,

[TABLE]

We emphasize that this limit of $4$ TeV relies on the dielectron plus dimuon data with invariant mass in the $500-6000$ GeV mass range as recommended, with the cuts in transverse energy and momentum as recommended by ATLAS collaboration in [74, 75]. The projected bound of $6.4$ TeV assumes a similar detector with the same trigger efficiency running at $14$ TeV center-of-mass energy and with $3ab^{-1}$ of integrated luminosity. This lower mass bound of $4$ TeV is rather robust and important because the $Z^{\prime}$ is tied to the scale of symmetry breaking of the model. Since $m_{Z^{\prime}}=0.4v_{\chi}$ (see Eq.(2.33)), we automatically find $v_{\chi}>10$ TeV. Furthermore, using the mass relations between the $W^{\prime}$ and $Z^{\prime}$ bosons one finds a lower mass bound on the $W^{\prime}$ mass which is much restrictive than other [76, 77] on its mass. Anyway, the $Z^{\prime}$ bound aforementioned is the most relevant to our discussion. We highlight that this limit has nothing to do with lepton flavor violation, it is rather simply based on searches for heavy dilepton resonances. Therefore, they are not relevant to our discussion because the mass of the scalars that enter in the lepton flavor violation discussion are not much sensitive to the energy scale of 3-3-1 symmetry breaking. Nevertheless, we emphasize that our entire discussion of lepton flavor violation is fully consistent with these bounds on the $Z^{\prime}$ mass. Now we will address collider searches for lepton flavor violation in what follows.

3.2 Searches for a Doubly Charged Scalar at the LHC

The presence of a doubly charged scalar in the spectrum is typical signature of a type II seesaw mechanism [78, 79, 80, 81, 82, 83, 84, 85, 86]. Typically, this type II seesaw is realized by the addition of a scalar triplet. This popular extension triggered several phenomenological analyses. As we mentioned previously, the scalar triplet that arises after the spontaneous symmetry breaking is key to the type II seesaw mechanism in our model. We emphasize, however, that in our model we have a type I + II seesaw because we do also have Dirac neutrino masses. After spontaneous symmetry breaking, we can single out the Yukawa term involving the scalar triplet and SM particles which reads [55],

[TABLE]

where $L$ is the SM lepton doublet with,

[TABLE]

where $S^{0}\equiv S^{0}_{11}$, $S^{-}\equiv S^{-}_{12}$, $S^{--}\equiv S_{22}^{--}$, as defined in Eq.(2.20).

The doubly charged scalar decay width into charged leptons is found to be [87],

[TABLE]

where $x=1$ for $i\neq j$ and $x=2$ for $i=j$. Thus is clear that the branching ratio into charged leptons can change depending on the Yukawa couplings. Different choices lead to different branching ratios and consequently different lower mass bounds. The dominant decay mode determines cuts, detector efficiency, and backgrounds which the signal is subject to, and consequently yielding different lower mass bounds. Dielectron and dimuon channels offer a cleaner environment and thus yield stronger bounds. If lepton flavor violation is assumed the SM background is suppressed, which again strengthens the limits. This is reasoning behind the limits derived using LHC data.

We used the code fastlim described in [88], and adopted a parton distribution function at next-to-next leading [89, 90] in order to project the LHC sensitivity for a high-luminosity (HL) setup, following the recommendations presented in [91]. We stress that the HL-LHC limit refers to a detector similar to LHC running at $14$ TeV with $3~{}ab^{-1}$ of integrated-luminosity. The High-Energy LHC configurations represents a $27$ TeV colliding beam with $15~{}ab^{-1}$ of data. We highlight that these bounds are based on the simulated signal $q\bar{q}\rightarrow Z,\gamma\rightarrow\phi^{++}\phi^{--}$ as outlined in [87], which features bounds stronger than previous previous studies [92, 93, 94]. In summary we derived,

[TABLE]

The latest LHC search for doubly charged scalars was with $12.9~{}fb^{-1}$ of data, but notice that with $36~{}fb^{-1}$ we expect LHC to already rule out doubly charged scalars with masses around $940$ GeV, assuming a normal mass ordering for the active neutrinos. If we had considered an inverted mass ordering a weaker bound would have been found, lying around $900$ GeV. This difference in the lower mass bound from LHC for normal and inverted ordering will not cause a meaningful impact on our conclusions and with this understanding in mind, we will simply quote those in Eq.(3.5). There are other important limits on this scenario [95, 96, 83, 97, 98] but the ones we quote are the most relevant. For a more complete discussion of the collider bounds on the type II seesaw we refer to [99].

Moreover, it is exciting to see that HE-LHC can potentially probe doubly charged scalar with masses up to $\sim 5$ TeV. These limits are quite important and serve as an orthogonal test to the type I + type II seesaw scenario we are investigating because that restricts the region in which the seesaw mechanism is viable. Having in mind that the doubly charged scalar is key to the lepton flavor violation observables we are about to discuss, such collider bounds stand as a complementary and important cross-check to lepton flavor violation signatures.

4 Lepton Flavor Violation

Lepton flavor violation is one of the most interesting probes of physics beyond the SM. The main lepton flavor violation signatures of the seesaw mechanism stem from muon decay namely, $\mu\rightarrow e\gamma$ and $\mu\rightarrow 3e$. There are other sources of lepton flavor violation such as $\mu-e$ conversion but they are subdominant [100, 101, 102, 103]. Other lepton flavor violating decays involving the $\tau$ lepton are less promising, unless one invokes a mechanism to significantly suppress $\mu\rightarrow e\gamma$ [104, 105, 106, 107, 108, 109, 110, 111, 112, 113]. Anyway, going back to the relevant muon decays, $\mu\rightarrow e\gamma$ and $\mu\rightarrow 3e$, one can check that the current bounds read $BR(\mu\rightarrow e\gamma)<4.2\times 10^{-13},BR(\mu\rightarrow 3e)<10^{-12}$, and future experiments aim $BR(\mu\rightarrow e\gamma)<4\times 10^{-14},BR(\mu\rightarrow 3e)<10^{-16}$. Therefore, we expect an important experimental improvement in the near future. Eventually, we will superimpose these limits with the model’s contribution. That said, having in mind that we have a dominant type II seesaw setup, the first contribution to $\mu\rightarrow e\gamma$ in our models stems from 1-loop processes involving the doubly and singly charged scalars which lead to,

[TABLE]

where $\alpha_{EM}$ is the fine-structure constant, $G_{F}$ the Fermi constant, $m_{S^{\pm\pm}}$ the mass of doubly charged scalar, $m_{S^{\pm}}$ the mass of singly charged scalar. In what follows will assume that the doubly charged and singly charged scalar have the same mass. This assumption will allow us to connect $\mu\rightarrow e\gamma$ directly to $\mu\rightarrow$ 3e and LHC limits on the doubly charged scalar mass.

There is an additional source of lepton flavor violation that rises from charged current which reads,

[TABLE]

where $W^{\prime}$ , is a charged gauge boson defined in Eq.(2.31) and has a mass equal to $0.3v_{\chi}$, $v_{\chi}$ being the energy scale of 3-3-1 symmetry breaking. This interaction results in the following branching ratio,

[TABLE]

where,

[TABLE]

with $g_{v}$, $g_{a}$ being the couplings constants that encompass the constants in Eq.(4.2) and the neutrino mixing matrices, and $I_{f,\,3}^{\pm\,\pm}$ given by,

[TABLE]

As one can see, it is not obvious the computation of the $\mu\rightarrow e\gamma$ decay in our model. In the regime which the right-handed neutrinos have identical masses to the $W^{\prime}$, the prediction for $\mu\rightarrow e\gamma$ is greatly simplified yielding,

[TABLE]

where $g^{\nu_{R}e}=g/(2\sqrt{2})U^{\nu_{R}e}$ and $g^{\nu_{R}\mu}=g/(2\sqrt{2})U^{\nu_{R}\mu}$.

Keeping right-handed neutrino masses similar to the $W^{\prime}$ mass and around the weak scale one can have a visible $\mu\rightarrow e\gamma$ decay, but if right-handed neutrino masses are much larger than the TeV scale, one can plugging the numbers in Eq.(4.3) using Eq.(4.4)-(4) to show that ${\rm BR}(\mu\rightarrow e\gamma)<10^{-15}$, which is beyond reach current and projected experiments. On the other hand, even in this scenario where right-handed neutrinos are very heavy, the ${\rm BR}(\mu\rightarrow e\gamma)$ can be large due to the presence of a type II seesaw mechanism which features doubly charged and singly charged scalars contributions.

Bearing in mind that the right-handed neutrinos in our model will be heavy, and one can neglect the $W^{\prime}$ contribution to the $\mu\rightarrow e\gamma$ decay and focus on the seesaw component. That said, another important observable is the $\mu\rightarrow 3e$ decay [73]. Since only the doubly charged Higgs contributes to this decay the calculation is simpler and leads to,

[TABLE]

On one hand we can see that the Yukawa couplings $f^{\nu}_{ab}$ dictate the lepton flavor violation observables, but on the other hand these Yukawa couplings enter in the neutrino mass matrix. Therefore, neutrino masses and lepton flavor violation observables are correlated.

Going back to Eq.(2.28), if $v_{s13}^{2}/\Lambda\sim v_{s11}$ as we will assume throughout, we have a dominant type II seesaw setup with the active neutrino masses set by vacuum expectation value, $v_{s11}$, i,e. $(m_{\nu})_{ab}\simeq\sqrt{2}f^{\nu}_{ab}v_{s11}$. The picture is not so simple because neutrinos oscillate and we need to reproduce the oscillation pattern. Therefore, these Yukawa couplings are found to be [101],

[TABLE]

where U is the Pontecorvo, Maki, Nakagawa, Sakata (PMNS) neutrino mixing matrix of dimension $3\times 3$, parametrized as follows [30],

[TABLE]

with the mixing parameters as shown in Table 4.11. [114],

In summary, one needs to incorporate neutrino oscillations to have more solid predictions for the lepton flavor violation observables. We will explore this fact by investigating several benchmark points which encompass normal and inverted mass ordering and different absolute neutrino masses. We will show that the neutrino mass spectrum is rather relevant to the overall lepton flavor violation signatures, a fact that has not been explored in detail in the context of 3-3-1 models. With this input from neutrino oscillations, our reasoning goes as follows:

• •

We choose a neutrino mass ordering;

• •

Then we pick a neutrino mass $m_{\nu 1}$, which then basically fixes $m_{\nu 2}$ and $m_{\nu 3}$, for a given $vev$, $v_{s11}=1-100$ eV. From this, we find the Yukawa couplings that reproduce this spectrum taking into account the oscillation patterns using Eq.(4.9);

• •

With these Yukawa couplings we use Eq.(4.1)-(4.8) to compute the lepton flavor violating muon decays.

In Table 2 we summarize our findings using this logic. The table will allow the reader to easily follow our reasoning as we discuss the results in figure 1 where we display four of these benchmark points for $m_{\nu 1}=0.1$ eV and $m_{\nu 1}=0.01$ eV including normal and inverted mass ordering. The blue (red) lines are the model predictions for the muon decays for inverted (normal) neutrino mass hierarchies. The difference between the solid and dashed lines is the absolute mass for the neutrino flavor $\nu_{1}$. For instance, the red solid line in the left-panel of figure 1 accounts for the $BR(\mu\rightarrow e\gamma)$ for $m_{\nu 1}=0.01$ eV within a normal mass ordering. The gray region represents the region currently excluded by LHC based on the search for doubly charged scalars within a type II seesaw framework with normal mass ordering. The dashed and dotted vertical gray lines are the HL-LHC and HE-LHC projected exclusion limits. The black horizontal lines are the current and projected bounds on the $\mu\rightarrow e\gamma$ (left-panel) and $\mu\rightarrow 3e$ (right-panel) decays.

One can notice that the neutrino mass ordering has a great impact on the lepton flavor violating muon decays. Looking at the first benchmark scenario with $m_{\nu 1}=0.1$ eV and $v_{s11}=1$ eV, which assumes an inverted mass ordering (IO) we get BR$(\mu\rightarrow e\gamma)=0.024/m_{S^{\pm\pm}}^{4}$, BR$(\mu\rightarrow 3e)=7.5/m_{S^{\pm\pm}}^{4}$. Having in mind that the current experimental limits, we conclude that we can probe doubly charged scalars with masses of $600$ GeV and $1.3$ TeV using from $\mu\rightarrow e\gamma$ and $\mu\rightarrow 3e$ decays. It is exciting to see that in this setup $\mu\rightarrow 3e$ provides stronger bounds than the LHC.

However, the larger the vacuum $v_{s11}$ the smaller the Yukawa couplings needed to reproduce the same neutrino masses. Hence, when we set $v_{s11}=100$ eV, the predictions change drastically to BR$(\mu\rightarrow e\gamma)=2.4\times 10^{-10}/m_{S^{\pm\pm}}^{4}$, BR$(\mu\rightarrow 3e)=7.5\times 10^{-10}/m_{S^{\pm\pm}}^{4}$. For these scenarios where the vacuum of is much larger than $1$ eV, LHC constitute the best probe. For concreteness, in this second case described above, taking $m_{S^{\pm\pm}}=1000$ GeV, would lead to muon decays much smaller than current and projected sensitivity [73], making HL-LHC and HE-LHC the best laboratories, since HE-LHC will probe masses of about $\sim 5$ TeV, for instance. All these conclusions are quite visible in figure 1.

Considering the normal mass ordering (NO) we conclude that the qualitative statements do not change. Taking $m_{\nu 1}=0.1$ eV, $v_{s11}=1$ eV, quantitatively we notice that the while the inverted hierarchy gives BR$(\mu\rightarrow e\gamma)=0.024/m_{S^{\pm\pm}}$ we find BR$(\mu\rightarrow e\gamma)=0.44/m_{S^{\pm\pm}}$, which is a factor of 20 larger. A larger much decay into $3e$ is also found for the NO compared to the IO (see figure 1). An orthogonal way to look at this is by noticing that the region between the current and projected limits delimit a signal region of lepton flavor violation. Looking at both panels of figure 1 we conclude that doubly charged scalars with masses around $1-2$ TeV might be spotted at the $\mu\rightarrow e\gamma$ decay, whereas the $\mu\rightarrow 3e$ will be able to detect such scalars with masses of up to $\sim 30$ TeV. The experimental progress on the search for the $\mu\rightarrow 3e$ decay is remarkable and it will surpass even the HE-LHC regardless of the mass ordering for benchmark scenarios where $v_{s11}\sim 1$ eV.

In the left-panel of figure 1 one can clearly see the impact of changing the value of the neutrino masses in the $\mu\rightarrow e\gamma$ decay. This can be checked by comparing the ratio between the dashed blue and red lines with the solid blue and red lines. This is not true for the $\mu\rightarrow 3e$ decay though (see right-panel of figure 1) . Comparing the NO and IO predictions for $m_{\nu 1}=0.01$ eV and $v_{s11}=1$ eV, we find no much difference, this is because the Yukawa couplings relevant for this observable are similar regardless of the neutrino mass ordering. This would continue to be true if we had taken even smaller values for $m_{\nu 1}$ because when $m_{\nu 1}$ is sufficiently small, the measured mass differences of the neutrino flavors govern by the neutrino mixings and thus the $\mu\rightarrow 3e$ decay. When we take $m_{\nu 1}=0.1$  eV, then the difference in the predictions for NO and IO is noticeable. One can easily use Table 2 and figure 1 to validate our conclusions .

We can conclude that regardless of the mass ordering and absolute value of the active neutrino masses that HE-LHC will solidly probe the type II seesaw model with respective to its contributions to the $\mu\rightarrow e\gamma$ decay. Concerning the $\mu\rightarrow 3e$ decay the situation changes due to the fantastic experimental sensitivity aimed in the near future. The $\mu\rightarrow 3e$ decay will be able to probe this model for doubly charged scalar masses up to 30 TeV, which is way beyond HE-LHC reach, no matter the neutrino mass ordering. If the absolute neutrino masses are much smaller than $0.01$ eV, then $\mu\rightarrow 3e$ decay becomes smaller making HE-LHC still the best probe.

Our conclusions explicitly show the importance of searching for signs of lepton flavor violation in collider and muon decays. The conclusion about which probe yields stronger bounds depends strongly on the mass ordering adopted, the absolute neutrino masses and which much decay one considers. In the $1-5$ TeV mass region of the doubly charged scalar lepton flavor violation experiments and colliders offer orthogonal and complementary probes. Thus if a signal is observed in one of the two new physics searches, the other will be able to assess whether is stems from a seesaw framework.

In summary, within the 3-3-1 model with right-handed neutrinos a type I+II seesaw naturally emerges. In the scenario where we have a dominant type II seesaw, the model offers a clear prediction for the $\mu\rightarrow e\gamma$ and $\mu\rightarrow 3e$ decays. One may wonder how one could discriminate our model from other type II seesaw proposal and a plausible answer would go as follows: setting aside the type II seesaw, our model predicts the existence of $W^{\prime}$ and $Z^{\prime}$ gauge bosons, as well as heavy exotic quarks. The detection of multiple signals consistent with all these particles could serve a discriminator and favor our models over others.

5 Conclusions

We have discussed a model which promotes $SU(2)_{L}\times U(1)_{Y}$ to $SU(3)_{L}\times U(1)_{N}$. In this extended gauge sector, all fermions get masses via a spontaneous symmetry breaking mechanism that encompasses three scalar triplets but neutrinos. A scalar sextet is added to incorporate neutrino masses. After spontaneous symmetry breaking this scalar sextet breaks down to a doublet and scalar triplet, which play a role in the type I and type II seesaw mechanism. We focus on a scenario of type II seesaw dominance where the relevant lepton flavor violation observables namely, $\mu\rightarrow e\gamma$ and $\mu\rightarrow 3e$, are directly tied to neutrino mass ordering and collider bounds on the doubly charged scalar.

We have explicitly shown how the absolute mass scale and neutrino mass ordering change the model predictions for lepton flavor violation within the type II seesaw framework. Combining the LHC, HL-LHC, HE-LHC sensitivity to doubly charged scalars and the experimental sensitivity to these rare muon decays we concluded that for doubly charged scalar with masses around $1-5$ TeV, these probes are rather complementary. Moreover, regardless of the mass ordering HE-LHC is expected to solidly test any possible signal seen in these muon decays.

One may wonder how one could discriminate our model from other types II seesaw proposals and a plausible answer would rely on the existence of $W^{\prime}$ and $Z^{\prime}$ gauge bosons, as well as heavy exotic quarks, all predicted in our model. The detection of multiple signals consistent with all these particles could serve a discriminator and favor our models over others.

6 Acknowledgments

The authors thank Diego Cogollo for discussions. MMF acknowledges financial support from CNPq grant 308933/2015-0, and from FAPEMA agency from the grants FAPEMA/UNIVERSAL/00880/15 and FAPEMA/PRONEX 01452-14. SK acknowledges support from Fondecyt (Chile) grant No. 1190845 and CONICYT (Chile) PIA/Basal FB0821. TM thanks CAPES for the fellowship. FSQ acknowledges support from CNPq grant 303817/2018-6, UFRN, MEC and ICTPSAIFR FAPESP grant 2016/01343-7.

Appendix A Appendix

In this appendix will describe in more detail and pedagogical manner some sections of the model which are relevant to our reasoning.

A.1 Lepton Masses

The leptons masses could in principle be generated via the Yukawa lagrangian,

[TABLE]

where $h_{ab}^{\nu}$ is an antisymmetric constant coupling matrix and $f_{ab}^{\nu}$ is a symmetric constant coupling matrix. The first and third terms conserve the lepton flavor, while the second term violate it. Some the scalars inside the scalar sextet S do have lepton number, for this reason the third term conserves lepton number as well.

The second term is not problematic but will be removed because a set of discrete that will be invoked. This set of $Z_{2}$ symmetries are needed to avoid mixing between the SM quarks and the exotic ones. One of them requires $\rho\rightarrow−\rho$, which forbids the second term above. Although, we also need to impose $e_{R}\rightarrow-e_{R}$ to generate masses for charged leptons.

In particular, the charged lepton masses arise from,

[TABLE]

In this way, in the flavor space the Dirac mass matrix for the charged leptons, at tree level, is given by

[TABLE]

where $v_{\rho}$ is the vacuum expectation value of the neutral scalar $\rho_{0}$ and $h_{ab}^{l}$ is the coupling constant matrix.

The neutrino masses come from the first second term only when the neutral components of scalar sextet acquire a vacuum expectation value as follows,

[TABLE]

We may arrange the mass terms as,

[TABLE]

With this result one can now easily understand Eq.(2.23).

A.2 Yukawa Interactions of Quarks

We mentioned in the paper that we needed to invoke some discrete symmetries to prevent mixing between the SM and exotic quarks. We will explain this statement in more detail now.

The lepton number conserving terms in the renormalizable Yukawa Lagrangian for the quarks sector are

[TABLE]

while the lepton number violating terms of quarks are

[TABLE]

where $h$ and $s$ are constant couplings.

One might notice that the terms in Eq.(A.12) will give rise to mass mixing terms involving the SM and exotic quarks, which can be problematic because they will lead to changes in the properties of the SM quarks. Therefore, one needs to prevent that and to do so invoke some discrete symmetries. The discrete symmetries have to be such that keep all the desired mass terms for the SM quarks and neutrino masses but forbid these ones. The set of discrete symmetries is,

[TABLE]

where $\alpha=1,2;$ $a=1,2,3.$

From Eq. ((A.10)), after spontaneous symmetry breaking we find,

[TABLE]

Thus, we can write the SM quark mass as follows,

[TABLE]

which leads to,

[TABLE]

Similarly, for the down-type quarks we get,

[TABLE]

with,

[TABLE]

We can see that the the type-up and type-down quarks are associated with the VEVs $v_{\eta}$ and $v_{\rho},$ related to the electroweak scale.

From the Eq. (A.13), we can write the $U$ extra quark mass term

[TABLE]

From the Eq. (A.13), we can write the $D$ extra quark mass lagrangian

[TABLE]

The $D$ quark mass matrix in the basis ($D_{1},D_{2})$ is

[TABLE]

Notice that the masses of $D$ quarks will depend the VEV $v_{\chi},$ related to the TeV-scale.

A.3 Scalar Sector

Considering the three scalar fields ($\chi,$ $\eta,$ $\rho),$ the Higgs potential more general, renormalizable and invariant on the $SU(3)_{L}\otimes U(1)_{X}$ symmetry group is

[TABLE]

where $\mu_{i}$ are constants, $\lambda_{i}$ and $f$ are constant couplings. Furthermore, the additional terms of potential with sextet scalar, as combinations with the others scalar triplets, are

[TABLE]

The permitted terms of the scalar potential by the discrete symmetry are

[TABLE]

In this model, the scalar triplets develop non-trivial vacuum expectation values (VEV), in order to engender correct spontaneous symmetry breaking (SSB), as written below

[TABLE]

Observe that the scalar triplet $\chi$ develops VEV only on the third neutral component, while $\eta$ develops VEV only on the first neutral component. The steps of symmetry breaking transition is given by SU(3)_{L}\otimes U(1)_{X}\overset{\left\langle\chi\right\rangle}{\rightarrow}SU(2)_{L}\otimes U(1)_{Y},\while$\ \ SU(2)_{L}\otimes U(1)_{Y}\overset{\left\langle\eta\right\rangle,\left\langle\rho\right\rangle}{\rightarrow}U(1)_{Q},$ i.e., the triplet $\chi$ develops VEV breaking the $3-3-1$ gauge symmetry to SM, while the triplets $\eta$ and $\rho$ develop VEV breaking the SM gauge symmetry to the QED (Quantum Electrodynamics).

Regarding the scalar sextet, the three neutral components develop VEVs in the following way:

[TABLE]

We will see that $v_{s_{1},}$ $v_{s_{3}}$ and $\Lambda$ are responsible for the mass for the left-handed neutrinos, while $\Lambda$ is responsible for the right-handed neutrinos Dirac masses. After the SSB of $SU(3)_{L}\otimes U(1)_{X}$ to $SU(2)_{L}\otimes U(1)_{Y}$, the scalar sextet results in a triplet plus a doublet and a singlet ($\mathbf{6}\rightarrow\mathbf{3}+\mathbf{2}+\mathbf{1})$, as follows

[TABLE]

where

[TABLE]

In this model, the lepton number distribution of the scalars is

[TABLE]

We can see that $S_{12}^{-}$ and $S_{22}^{--}$ carry two lepton numbers. Both are essential in our analysis of charged lepton flavor violating decay of muon.

A.4 Charged Lepton Flavor Interactions

The Yukawa Lagrangian with charged lepton flavor violation (CLFV) is given by

[TABLE]

where $a,b=1,2,3$ indicates the lepton generations and $m,n=1,2,3$ indicates the entries of the sextet, $f_{ab}$ is symmetric. The CLFV interactions in the $\mu-e$ sector results from taking $a=1,b=2$ in the above equation,

[TABLE]

Using the following relations:

[TABLE]

we have,

[TABLE]

We see that these terms directly induce $\mu\rightarrow e\gamma$ lepton flavor violating decay mediated by the doubly charged scalar $S^{++}$. Proceeding in an analogous way, we can easily obtain the relevant CLFV terms mediated by the singly charged scalar $S^{+}$.

Appendix B Derivation of the $\mu\rightarrow e\gamma$ decay

The on-shell amplitude for the $l_{j}^{-}\rightarrow l_{i}^{-}\gamma$ process is written in the form

[TABLE]

where $\varepsilon^{\mu}$ is the polarization vector of the on-shell photon, $e$ is the electric charge, $u_{i}$ is the spinor of $l_{i}^{-},$ $u_{j}$ is the spinor of $l_{j}^{-},$ $m_{l_{j}}$ is the mass of $l_{j}$ lepton, $\sigma^{\mu\nu}$ is the comutator of $\gamma$ matrices, $q$ is the photon 4-momentum, $p$ is the $l_{j}$ lepton 4-momentum, $P_{L}$ is the chirality projector of the left-handed lepton, $P_{R}$ is the chirality projection of the right-handed lepton, $A_{2}^{L}$ and $A_{2}^{R}$ are coupling constants.

The squared modulus of the amplitude is

[TABLE]

Summing over the polarization states of the on-shell photon, we have the squared amplitude written in the form

[TABLE]

Using the completeness relation $\sum_{\lambda}\varepsilon_{\mu}^{\lambda}\varepsilon_{\alpha}^{\ast\lambda}=-g_{\mu\alpha}$, we get

[TABLE]

In order to obtain the squared amplitude explicitly, we need to calculate the term

[TABLE]

where

[TABLE]

where $P_{L}=\left(1-\gamma_{5}\right)/2$ and $P_{R}=\left(1+\gamma_{5}\right)/2$ are the chirality projectors.

By using the following expression

[TABLE]

the Eq. (B.7) can be written as

[TABLE]

Reminding that $P_{L}^{\dagger}=P_{L}$ and $P_{R}^{\dagger}=P_{R},$ so we get

[TABLE]

Then, we have the following result

[TABLE]

Given that $P_{L}\gamma^{0}=\gamma^{0}P_{R}$ and $P_{R}\gamma^{0}=\gamma^{0}P_{L},$ we have

[TABLE]

Therefore the squared amplitude is written in the following form

[TABLE]

Summing over the spins of the fermions and writing the multiplication matrix in index form, we get

[TABLE]

Using the completeness relation

[TABLE]

and doing the average of the initial spin of fermion, we get

[TABLE]

We need to put back the equation above into normal matrix multiplication order

[TABLE]

We can write the equation above in terms of a trace

[TABLE]

Using $\left\{\gamma_{5},\gamma^{\mu}\right\}=0$ and $P_{L}^{2}=P_{L},$ $P_{R}^{2}=P_{R},$ $P_{L}P_{R}=P_{R}P_{L}=0,$ we have

[TABLE]

Let’s calculate, ignoring $m_{1},$ the following term

[TABLE]

Now we need to calculate this equation below

[TABLE]

Let’s calculate explicitly the equation above:

[TABLE]

Inserting the $g_{\mu\alpha}$,

[TABLE]

[TABLE]

Writing \hbox to0.0pt{\hbox{\mskip 1.0mu/}\hss}p_{2}=p_{2\eta}\gamma^{\eta}, we get

[TABLE]

Let’s simplify the term above, using the contraction identities

[TABLE]

It leads us to the result

[TABLE]

Now, plugging $g_{\mu\alpha}$ in the expression,

[TABLE]

We can written the equation above as

[TABLE]

It follows that

[TABLE]

Firstly, in order to simplify the evaluation, let’s calculate the term only with \hbox to0.0pt{\hbox{\mskip 1.0mu/}\hss}p_{1}, as follows below

[TABLE]

Let’s calculate the term in the braces explicitly

[TABLE]

With \hbox to0.0pt{\hbox{\mskip 1.0mu/}\hss}p_{1}=p_{1\alpha}\gamma^{\alpha}, we get

[TABLE]

Putting in evidence $p_{1\alpha}$ in the equation above, it leads to

[TABLE]

Taking the trace, we have

[TABLE]

Observing that the trace of an odd product of $\gamma_{\nu}$vanishes and using the trace identities,

[TABLE]

we get the result,

[TABLE]

Inserting the product of photon 4-momenta $q_{\nu}q_{\beta},$ we have

[TABLE]

Taking the contractions in the equation above, we get

[TABLE]

Given that

[TABLE]

it follows,

[TABLE]

Analogously to what we calculated above, now we will proceed onto the evaluation of the term with \hbox to0.0pt{\hbox{\mskip 1.0mu/}\hss}q:

[TABLE]

With \hbox to0.0pt{\hbox{\mskip 1.0mu/}\hss}q=q_{\mu}\gamma^{\mu}, we get

[TABLE]

Taking the trace of the equation above

[TABLE]

we get

[TABLE]

Using the trace identities, we have

[TABLE]

Plugging the $q_{\nu}q_{\beta}$ in the equation above, we get

[TABLE]

where $q^{\mu}q_{\mu}=q.q=q^{2}=0$ (for the case of on-shell photon).

Having in mind the results we obtain the follwing amplitude,

[TABLE]

where $m_{l_{j}}=m_{\mu}$ (It’s the mass of the decaying particle, i.e., the muon mass). This result is essential for the obtaining of the decay rate, since it depends on the spin-averaged squared amplitude.

In the centre of the mass of the system, the decaying particle is at rest, then $E=m_{2}=m_{\mu}$ and $\text{p}_{\mu}=0$ $\left(\text{3-momentum of the muon}\right).$ The 3-momenta of the final state particles are $\text{p}_{\gamma}=-\text{p}^{\ast}$ and $\text{p}_{e}=\text{p}^{\ast}.$ The 4-momentum of the $\mu^{-},$ $e^{-}$ and $\gamma$ are respectively,

[TABLE]

Here we are using the notation $p^{\alpha}=p$ (for the contravariant 4-vector) .

Taking $E_{e}=\text{p}^{\ast}$ and $E_{\gamma}=\text{p}^{\ast},$ we can written the 4-momenta relatives to the centre of mass frame as:

[TABLE]

The magnitude of 3-momentum of the final-state particles (in the centre of mass frame) is

[TABLE]

where we consider $m_{\mu}\gg m_{e}$, i.e., we neglect the electron mass in relation to the muon mass.

In order to write the squared spin-averaged amplitude in a more simplified way, we can write the product of two contractions as,

[TABLE]

Substituting the result above in the Eq. (B.53), the squared amplitude (averaged over particle spin) is written as

[TABLE]

Having in hands the squared amplitude, we can obtain the decay rate of the charged lepton flavor violating process for the muon decay in electron plus one photon. For this, we can need to write the decay rate for the two-bodies decay, which is given by

[TABLE]

Using Eq. (B.56) for $p^{*}$, the decay rate is

[TABLE]

Now substituting the squared spin-averaged amplitude given in the Eq. (B.58), we obtain

[TABLE]

Therefore, the decay rate of the charged lepton flavor violating decay of the muon is

[TABLE]

Observe above that the rate decay of charged lepton flavor violating muon depends on the mass muon, the electric charge and it is proportional to the product of sum of the squares of $A_{2}^{L}$ and $A_{2}^{R}$.

It’s well known that the decay rate for the muon decay $\left(\mu\rightarrow e\overline{\nu}_{e}\nu_{\mu}\right)$, in the SM, is

[TABLE]

where $G_{F}$ is Fermi constant and $m_{\mu}$ is the muon’s mass. This rate decay of the muon does not violate the charged lepton flavor.

The branching ratio for the $\left(\mu\rightarrow e\overline{\nu}_{e}\nu_{\mu}\right)$ charged lepton flavor violating process is

[TABLE]

Thus, given that $\alpha=e^{2}/4\pi$ (in natural units), we have finally the branching ratio of the charged lepton flavor violating muon decay for the ($\mu\rightarrow e\gamma)$ process

[TABLE]

where $\alpha$ is the electromagnetic fine-structure constant, $G_{F}$ is the Fermi constant. Besides to $\mu\rightarrow e\gamma$ decay, this branching ratio can be used for others decay processes, as $\tau\rightarrow\mu\gamma$.

It’s important to point that the branching ratio, given by the Eq. (B.66), is very general, since we can use it for various models, for instance: supersymmetric and $3-3-1$ models. In fact, we need to obtain only $A_{2}^{L,R}$ for each model with its specific contributions related on the particle content.