Lepton masses and mixing in a scotogenic model

TL;DR

This paper proposes an extended three-Higgs-doublet model with discrete symmetries that explains lepton masses, mixing, and dark matter candidates, while analyzing experimental constraints and predictions for rare decays.

Contribution

It introduces a novel scotogenic model with specific symmetries, calculating mass spectra and linking dark matter candidates with leptonic decay constraints.

Findings

Identified scalar and pseudoscalar dark matter candidates.

Predicted signals for $ ext{mu} o e ext{gamma}$ decay within experimental reach.

Connected decay constraints to predictions for $ ext{mu} o eear{e}$ decay.

Abstract

We consider an extension of the standard model with three Higgs doublet model and $S_3\times \mathbb{Z}_2$ discrete symmetries. Two of the scalar doublets are inert due to the $\mathbb{Z}_2$ symmetry. We have calculated all the mass spectra in the scalar and lepton sectors and accommodated the leptonic mixing matrix as well. We also show that the model has scalar and pseudoscalar candidates to dark matter. Constraints on the parameters of the model coming from the decay $\mu\to e\gamma$ were considered and we found signals between the current and the upcoming experimental limits, and from that decay we can predict the one-loop $\mu\to ee\bar{e}$ channel.

Tables

Table 1. Table 1: Transformation properties of the fermion and scalar fields under S 3 subscript 𝑆 3 S_{3} and ℤ 2 subscript ℤ 2 \mathbb{Z}_{2} symmetries. Quarks and charged leptons are singlets of S 3 subscript 𝑆 3 S_{3} and even under ℤ 2 subscript ℤ 2 \mathbb{Z}_{2} .

Symmetry	$L_i$	$l_{jR}$	$N_s$	$N_d$	$S$	$D$	${\zeta }_d$
$S_3$	1	1	$1^{\prime }$	2	1	2	2
${\mathbb{Z}}_2$	1	1	-1	1	1	1	-1

Table 2. Table 2: Masses of the scalars in Scotogenic model (in GeV).

$V_L^l$ parameterization	Masses in TeV	$G_{11}^l$	$G_{12}^l$	$G_{13}^l$	$G_{22}^l$	$G_{23}^l$	$G_{33}^l$
P1	$M_R=2.8$, $M_0=2.2$	0.000421836	0.000514741	-0.000800772	0.00374767	-0.00356758	0.00367645
P2	$M_R=2$, $M_0=2.2$	0.000421864	0.000514825	-0.000800852	0.00374768	-0.00356756	0.00367641
P3	$M_R=1.5$, $M_0=2.2$	0.000421896	0.000514922	0.000800945	0.0037477	-0.00356754	0.00367636
P4	$M_R=1$, $M_0=2.2$	0.00042195	0.000515086	-0.000801103	0.00374772	-0.00356751	0.00367628
P5	$M_R=0.5$, $M_0=2.2$	0.00042205	0.000515393	-0.000801398	0.00374778	-0.00356744	0.00367612

Table 3. Table 3: The Yukawas values for Eq. ( 19 ) that solve DM.

Scenario	$G_{es}^{\nu }$	$G_{\mu s}^{\nu }$	$G_{\tau s}^{\nu }$	$G_{ed}^{\nu }$	$G_{\mu d}^{\nu }$	$G_{\tau d}^{\nu }$
1	1.17$\times {10}^{-7}$	${10}^{-11}$	${10}^{-11}$	${10}^{-11}$	${10}^{-11}$	2.81$\times {10}^{-7}$
2	1.56$\times {10}^{-7}$	${10}^{-11}$	${10}^{-11}$	${10}^{-11}$	${10}^{-11}$	3.75$\times {10}^{-7}$

Table 4. Table 4: Masses (in GeV) of the scalars in this model. a 6 subscript 𝑎 6 a_{6} is a dimensionless coupling in the non-Hermitian quartic scalar interaction which transfer the L 𝐿 L violation to the active neutrino sector.

Scenario	$M_{+1}$	$M_{+2}$	$m_{\zeta 1}$	$m_{\zeta 2}$	$M_{R1}$	$M_{R2}$	$M_{I1}$	$M_{I2}$	$M_d$	$M_s$	$a_6$
1	109.02	1477.64	749.81	3747.26	85.20	2085.74	161.84	2090.27	240	240	-1.16$\times {10}^{-1}$
2	109.12	1477.65	749.81	3747.26	257.25	2099.82	113.70	2087.10	240	240	2.6$\times {10}^{-1}$

Table 5. Table 5: l i → l j ​ γ → subscript 𝑙 𝑖 subscript 𝑙 𝑗 𝛾 l_{i}\to l_{j}\gamma , experimental upper limits and the SM predictions.

Decay	Current limit	Future limit	SM
Br($\mu \to e\gamma$)	TheMEG:2016wtm	Baldini:2013ke	${10}^{-48}$
Br($\tau \to e\gamma$)	Olive:2016xmw	Aushev:2010bq	${10}^{-49}$
Br($\tau \to \mu \gamma$)	Olive:2016xmw	Aushev:2010bq	${10}^{-49}$

Table 6. Table 6: Mass constraints for the right-handed neutrino with fixed values of m D + subscript 𝑚 superscript 𝐷 m_{D^{+}} in order to respect Br ( μ → e ​ γ ) Exp < 4.2 × 10 − 13 superscript → 𝜇 𝑒 𝛾 Exp 4.2 superscript 10 13 (\mu\to e\gamma)^{\text{Exp}}<4.2\times 10^{-13} , here G τ ​ d ν = G e ​ s ν ∼ 10 − 1 superscript subscript 𝐺 𝜏 𝑑 𝜈 superscript subscript 𝐺 𝑒 𝑠 𝜈 similar-to superscript 10 1 G_{\tau d}^{\nu}=G_{es}^{\nu}\sim 10^{-1} and G e ​ d , μ ​ d , μ ​ s , τ ​ s ν ∼ 10 − 5 similar-to superscript subscript 𝐺 𝑒 𝑑 𝜇 𝑑 𝜇 𝑠 𝜏 𝑠 𝜈 superscript 10 5 G_{ed,\mu d,\mu s,\tau s}^{\nu}\sim 10^{-5} .

$\mu \to e\gamma$
$m_{D^{+}}$ [GeV]	$m_N$ [GeV]
	$V_L^l$ parameterization A	$V_L^l$ parameterization B
80	$>$1140	$>$2610
250	$>$1000	$>$2510
500	$>$ 710	$>$2300
750	$>$ 295	$>$2040

Table 7. Table 7: Br ( μ → e ​ e ​ e ¯ ) → 𝜇 𝑒 𝑒 ¯ 𝑒 (\mu\to ee\bar{e}) predictions from the Br ( μ → e ​ γ ) → 𝜇 𝑒 𝛾 (\mu\to e\gamma) constraints.

Br$\left(\mu \to ee\overline{e}\right)$
$V_L^l$ parameterization A	$V_L^l$ parameterization B

Equations

$$-{\mathcal{L}}_{Yukawa}^{leptons}$$

$$S=\left(\begin{array}[]{c}S^{+}\\ \frac{1}{\sqrt{2}}(v_{SM}+\textrm{Re}S^{0}+i\textrm{Im}S^{0})\end{array}\right),\quad D_{1,2}=\left(\begin{array}[]{c}D^{+}_{1,2}\\ \frac{1}{\sqrt{2}}(\textrm{Re}D^{0}_{1,2}+i\textrm{Im}D^{0}_{1,2}).\end{array}\right),$$

$$V_{S_3}$$

$$V(S,D,{\zeta }_d)=V^{(2)}+V^{(4a)}+V^{(4b)}+V^{(4c)},$$

$$V^{(2)}$$

$$V^{(4a)}$$

$$V^{(4b)}$$

$$V^{(4c)}$$

$$v_{SM}[2{\mu }_{SM}^2+2a_4{v}_{SM}^2+b_1(v_1^2+v_2^2)]$$

$$v_1[4{\mu }_{\zeta }^2+2b_1{v}_{SM}^2+4c(v_1^2+v_2^2)+4{\mu }_{12}^2\frac{v_2}{v_1}]$$

$$v_2[4{\mu }_{\zeta }^2+2b_1{v}_{SM}^2+4c(v_1^2+v_2^2)+4{\mu }_{12}^2\frac{v_1}{v_2}]$$

$$v_{SM}[{\mu }_{SM}^2+a_4{v}_{SM}^2+b_1{v}_{\zeta }^2]=0,v_{\zeta }[2{\mu }_{\zeta }^2+b_1{v}_{SM}^2+4c{v}_{\zeta }+2{\mu }_{12}^2]=0.$$

$$a_4\ge 0,\frac{a_5}{2}\ge 0,C\ge 0,b_1\ge 0,a_1+a_3\ge 0,$$

$$b_1+b_2\ge 0,B\ge 0,c_1+c_2\ge 0.$$

$$\mathcal{M}^{2}_{1R}=\left(\begin{array}[]{ccc}2a_{4}v^{2}_{s}&b_{1}v_{SM}v_{\zeta}&b_{1}v_{SM}v_{\zeta}\\ &-\mu^{2}_{12}+2cv^{2}_{\zeta}&\mu^{2}_{12}+2cv^{2}_{\zeta}\\ &&-\mu^{2}_{12}+2cv^{2}_{\zeta}\end{array}\right).$$

$$m_1^2=-2{\mu }_{12}^2,$$

$$m_2^2=\frac{1}{2}\left[a_4{v}_{SM}^2+2c{v}_{\zeta }^2-\sqrt{a_4^2{v}_{SM}^4+2(b_1^2-2a_{4}c)v_{\zeta }^2{v}_{SM}^2+4c^2{v}_{\zeta }^2}\right],$$

$$m_3^2=\frac{1}{2}\left[[4a_4{v}_{SM}^2+8c{v}_{\zeta }^2{]}^2+\sqrt{a_4^2{v}_{SM}^4+2(b_1^2-2a_{4}c)v_{\zeta }^2{v}_{SM}^2+4c^2{v}_{\zeta }^2}\right].$$

$$\mathcal{M}^{2}_{2R}=\left(\begin{array}[]{cc}\mu^{2}_{d}+\frac{a^{\prime}}{2}v_{SM}^{2}+b_{2}v^{2}_{\zeta}&b_{3}v^{2}_{\zeta}\\ &\mu^{2}_{d}+\frac{a^{\prime}}{2}v_{SM}^{2}+b_{2}v^{2}_{\zeta}\\ \end{array}\right),$$

$${\mathrm{m}}_{R1}^2=2{\mu }_d^2+a^{\prime }{v}_{SM}^2+2(b_2-b_3)v_{\zeta }^2,{\mathrm{m}}_{R2}^2=2{\mu }_d^2+a^{\prime }{v}_{SM}^2+2(b_2+b_3)v_{\zeta }^2.$$

$$M_{I}=\left(\begin{array}[]{cc}\mu^{2}_{d}+\frac{a^{\prime\prime}}{2}v_{SM}^{2}+b_{2}v^{2}_{\zeta}&b_{3}v^{2}_{\zeta}\\ &\mu^{2}_{d}+\frac{a^{\prime\prime}}{2}v_{SM}^{2}+b_{2}v^{2}_{\zeta}\end{array}\right),$$

$$m_{I1}^2=2{\mu }_d^2+a^{\prime \prime }{v}_{SM}^2+2(b_2-b_3)v_{\zeta }^2,m_{I2}^2=2{\mu }_d^2+a^{\prime \prime }{v}_{SM}^2+2(b_2+b_3)v_{\zeta }^2.$$

$$M^{2}_{C}=\left(\begin{array}[]{cc}\mu^{2}_{d}+\frac{a_{5}}{2}v_{SM}^{2}+b_{2}v^{2}_{\zeta}&b_{3}v^{2}_{\zeta}\\ &\mu^{2}_{d}+\frac{a_{5}}{2}v_{SM}^{2}+b_{2}v^{2}_{\zeta}\end{array}\right),$$

$$m_{+1}^2={\mu }_d^2+\frac{a_5}{2}{v}_{SM}^2+(b_2-b_3)v_{\zeta }^2,m_{+2}^2={\mu }_d^2+\frac{a_5}{2}{v}_{SM}^2+(b_2+b_3)v_{\zeta }^2,$$

$$m_{R1}^2-m_{+1}^2$$

$$m_{R2}^2-m_{+2}^2$$

$$(M_{\nu }{)}_{ij}=\sum _{a,k}\frac{Y_{ik}{Y}_{jk}{M}_k}{32{\pi }^2}\left[\frac{m_{Ra}^2}{m_{Ra}^2-M_k^2}\ln \frac{m_{Ra}^2}{M_k^2}-\frac{{\mathrm{m}}_{Ia}^2}{m_{Ia}^2-M_k^2}\ln \frac{m_{Ia}^2}{M_k^2}\right],$$

$$(M_{\nu }{)}_{ij}$$

$$|V_{PMNS}|\approx\left(\begin{array}[]{ccc}0.815&0.565&0.132\\ 0.479&0.527&0.702\\ 0.327&0.635&0.700\\ \end{array}\right),$$

$$|V_{PMNS}|\approx\left(\begin{array}[]{ccc}0.795-0.846&0.513-0.585&0.126-0.178\\ 0.4205-0.543&0.416-0.730&0.579-0.808\\ 0.215-0.548&0.409-0.725&0.567-0.800\\ \end{array}\right),$$

$$\frac{d{n}_R}{dt}+3H{n}_R=-⟨\sigma |v|⟩[(n_R{)}^2-(n_R^{eq}{)}^2],$$

$$n_R^{eq}=g{\left(\frac{m_{R}T}{2\pi }\right)}^{3/2}\exp \left(-\frac{m_R}{T}\right),$$

$${\Omega }_R{h}^2\approx \frac{1.04\times 10^9{x}_F}{M_{Pl}\sqrt{g_{\ast }}(a+3b/x_F)},$$

$$x_F=\ln \left[\tilde{c}(\tilde{c}+2)\sqrt{\frac{45}{8}}\frac{g{m}_R{M}_{Pl}(a+6b/x_F)}{2{\pi }^3\sqrt{g_{\ast }(x_F)}}\right],$$

$$V^{\nu}_{L}\approx\left(\begin{array}[]{ccc}0.00010637&-1&0.0000444397\\ -1&-0.000106374857&0.00007634160462\\ 0.000076336876851&0.000044447836932&1\end{array}\right),$$

Full text

I Introduction

Although since 2012 we know that there exist a neutral spin-0 resonance with properties (mass and couplings) that are compatible, within the experimental error, with the Higgs boson of the standard model (SM) Aad:2012tfa ; Chatrchyan:2012xdj , the data do not exclude the existence of more scalar fields and almost all extensions of the SM include extra Higgs doublets. This is the reason for considering multi-Higgs models. Moreover, although many scalar doublets may exist in nature, it is possible that only one of them is the responsible for the electroweak spontaneous symmetry breaking and the generation of the charged fermion masses. In this case, the other scalar multiplets may be inert ones: they do not couple to fermions, do not contribute to the vector bosons masses, and interact only with vector bosons and with other scalars. This possibility was put forward many years ago in Ref. Deshpande:1977rw in which a $\mathbb{Z}_{2}$ symmetry was imposed to keep inert one of the doublets.

On the other hand, an indication that there must be physics beyond the SM is the origin of the neutrino masses. In fact, the generation of masses smaller than 0.1 eV demand the introduction of new degrees of freedom even in the context of the gauge symmetries of the SM. An interesting possibility is that the extra scalar fields that may exist as an extension of the SM also induce the appropriate neutrino mass. In particular, neutrino mass generation in a model with two doublets, being one of them inert, was considered by Ma Ma:2006km . This is the so called scotogenic mechanism for generating neutrino masses through one-loop corrections involving the inert neutral components.

One interesting feature of the mechanism is that it includes by construction one or more dark matter (DM) candidates, or the implementation of the baryon asymmetry in the Universe, relating in this way three of the more important problems in elementary particle physics: the generation of the neutrino masses, the nature of the DM, and the observed asymmetry between matter and anti-matter, see Ref. Aoki:2014lha and references therein. Moreover, the existence of many components of DM is interesting by their own. In this case DM may decay from heavier to lighter components, and also co-annihilate in two dark particles Dienes:2014via . In fact, this might be the only possibility to accommodate several astrophysical observations. For instance, the electron/positron excesses observed by many experiments and recently confirmed by AMS-02 Accardo:2014lma and the excess of gamma rays peaking at energies of several GeV from the region surrounding the Galactic Center Goodenough:2009gk ; Daylan:2014rsa must need at least two component DM. Moreover, the latter case can avoid some constraints on the one component DM from the AMS-02 data Geng:2014dea . Notwithstanding there are alternative interpretations for these gamma rays excess, see Abazajian:2015raa ; Fermi-LAT:2016uux . It is interesting that the one-doublet inert model has at least two DM components. However see Liu:2017drf .

Here we will work out a similar mechanism but in the context of the model with two inert scalar doublets proposed in Ref. Machado:2012ed . Moreover, the inert character is due to the $\mathbb{Z}_{2}$ symmetry, and the $S_{3}$ symmetry makes the scalar potential more predictive and easier to be analyzed. Although three right-handed neutrinos are introduced, the active neutrino masses do not arise through the type-I seesaw mechanism but via the scotogenic mechanism, at the 1-loop level. We need to add two real singlet scalar fields in order to accommodate the charged lepton masses and the Pontecorvo-Maki-Nakagawa-Sakata (PMNS) mixing matrix. In the context of inert doublet models, the latter issue is to the best of our knowledge done for the first time.

The outline of this paper is as follows. In the next section we discuss the model while in Sec. III analyse the scalar sector. Lepton mass matrices and the leptonic mixing matrix is shown in Sec. IV. In Sec. V we show that the model provides a multi-component DM spectra, although we do not consider the most general case. In Sec. VI we consider the decays $\mu\to e\gamma$, Subsec. VI.1, and $\mu\to e\bar{e}e$ in Subsec. VI.2. Our conclusions appear in Sec. VII.

II The Model

In Ref. Machado:2012ed it was proposed an extension of the electroweak SM with three Higgs scalars $H_{1,2,3}$ transforming as doublets under $SU(2)$ and having $Y=+1$. One $H_{1}\equiv S$ transforms as singlet of $S_{3}$, and the others as doublets, $D=(H_{2},H_{3})~{}\equiv~{}(D_{1},D_{2})$. Here we will extend the model of Ref. Machado:2012ed by adding three right-handed sterile neutrinos, $N_{1R}$ transforming as singlet, and $N_{d}=(N_{2R},N_{3R})$ transforming as doublets of $S_{3}$, and two real scalar singlets of $SU(2)$ ($Y=0$) but doublets of $S_{3}$, $\zeta_{d}=(\zeta_{1},\zeta_{2})$. See the other quantum numbers in Table 1. The vacuum alignment is given by $\langle S\rangle=v_{SM}/\sqrt{2}$, and $\langle D_{1},D_{2}\rangle=0$, $\langle\zeta_{1,2}\rangle=v_{\zeta}$. In the charged lepton and quark sectors, all usual fields of SM transform as singlet under $S_{3}$.

With these fields the Yukawa interactions in the lepton sector, invariant under the gauge, $S_{3}$ and $\mathbb{Z}_{2}$ symmetries (see Table 1) are given by

[TABLE]

where $i,j=e,\mu,\tau$ (we omit summation symbols), $L_{i}(l_{iR})$ and $N_{s,d}$ denote the usual left-handed lepton doublets (right-handed singlets) and the right-handed neutrinos, respectively; $[\tilde{D}\,\zeta_{d}]_{1^{\prime}}=\tilde{D}_{1}\zeta_{2}-\tilde{D}_{2}\zeta_{1}$. $[N_{d}\,D]_{1}=N_{2R}D_{1}+N_{3R}D_{2}$, according to the $S_{3}$ multiplication rules, and $\tilde{D}_{1,2}=i\tau^{2}D^{*}_{1,2}$. Notice that the doublets $D$ and $\zeta_{d}$ couple only with neutrinos. We assume that $\langle\zeta_{1,2}\rangle\lesssim\Lambda$, where $\Lambda$ is an energy scale much larger than the electroweak one. It is also interesting to note that the right-handed neutrinos in the $S_{3}$ doublet are mass degenerated, with mass $M_{d}$, which is different from the mass of the right-handed neutrino in singlet of $S_{3}$, which has a mass denoted by $M_{s}$. Notice that at three level active neutrinos are still massless.

III The scalar sector

The scalar sector of the model is presented as follows:

[TABLE]

plus the singlets $\zeta_{i}=v_{i}+\textrm{Re}\zeta_{i}+i\textrm{Im}\zeta_{i}$, $i=1,2$.

The scalar potential invariant under the gauge and $S_{3}\otimes\mathbb{Z}_{2}$ symmetries is

[TABLE]

with $\mu^{2}_{d}>0$ that is guaranteed by the $\mathbb{Z}_{2}$ symmetry.

We can write Eq. (3) explicitly as

[TABLE]

where

[TABLE]

where we have used $[\zeta_{d}\,\zeta_{d}]_{2^{\prime}}=(\zeta_{1}\zeta_{2}+\zeta_{2}\zeta_{1},\zeta_{1}\zeta_{1}-\zeta_{2}\zeta_{2})$. Notice that the term $\mu^{2}_{12}$ breaks softly the $S_{3}$ symmetry but not the $\mathbb{Z}_{2}$. Notice also that the $\mathbb{Z}_{2}$ symmetry forbids trilinear terms in the scalar potential $[D^{\dagger}\otimes D]_{1}\zeta_{i}$ and $\zeta^{3}_{i}$.

From derivation of Eq. (5), we obtain the constraint equations:

[TABLE]

where we have defined $c=c_{1}+c_{2}$. Notice from (6) that neither $v_{1}=0$ nor $v_{2}=0$ are allowed if $\mu^{2}_{12}\not=0$, hence we have that $v_{1}\not=0,v_{2}\not=0$, or $v_{1}=v_{2}\equiv v_{\zeta}$. We have chosen the latter case, so the constraint equations become

[TABLE]

The scalar potential has to be bounded from below to ensure its stability. In the SM it is easy to ensure the stability of this potential, we just have to ensure that $\lambda>0$. In theories in which the number of scalars is increased, it is more difficult to ensure that the potential is bounded from below, in all directions. A scalar potential has a quadratic form in the quadratic couplings, i.e. $A_{ab}\phi^{2}_{a}\phi^{2}_{b}$, where $\phi^{2}_{a}$ and $\phi^{2}_{b}$ represents the scalar fields, $S$, $D$ and $\zeta_{d}$. If the matrix $A_{ab}$ is copositive it is possible to ensure that the potential has a global minimum. Assuming a quadratic form (e.g. considering only the quartic terms of the potential) is valid, even if there exist trilinear terms, because in the case where the fields assume large values, the terms of order 2 and 3 are negligible compared to the terms of order 4. For more detail see Refs. ping ; Kannike:2012pe . We consider all quartic couplings positive, i.e. all $A_{ij}$ are positive. Below we will denote $B=a_{1}-2a_{2}-a_{3}$ and $C=a_{5}-2a_{6}+a_{7}$. And finally the following limits guarantee that the scalar potential is bounded the from below:

[TABLE]

Next, we consider the scalar mass spectra. In the $C\!P$-even sector the mass matrix becomes in block diagonal form with one $3\times 3$, $\mathcal{M}_{1R}$ sub-matrix and one $2\times 2$ matrix, $\mathcal{M}_{2R}$. The first one in the basis $(\textrm{Re}S^{0},\zeta_{1},\zeta_{2})$ is given by

[TABLE]

The respective eigenvalues are

[TABLE]

From $m^{2}_{1}$ we see that $\mu^{2}_{12}<0$, hence $m^{2}_{1}$ may have large mass. The SM-like Higgs boson may be identified with the scalar with mass $m_{3}$ in Eq. (10). In order to see this just make $v^{2}_{\zeta}=0$.

The second mass matrix $M_{2R}$ in the basis $(\textrm{Re}D^{0}_{1},\textrm{Re}D^{0}_{2})$ reads

[TABLE]

with the eigenvalues

[TABLE]

The mass matrix in the $C\!P$ odd sector has the form in the basis $(\textrm{Im}D^{0}_{1},\textrm{Im}D^{0}_{2})$ (the would-be Goldstone bosons has been already decoupled)

[TABLE]

with eigenvalues

[TABLE]

Above we have defined $a^{\prime}=a_{5}+a_{7}+2a_{6}$ and $a^{\prime\prime}=a_{5}+a_{7}-2a_{6}$.

The model allows four neutral scalars with different masses that could contribute to the DM relic density in different proportion: two $C\!P$ even and two $C\!P$ odd. Notice also that $a_{6}$ is the term in the scalar potential that transfer the $L$ violation to the active neutrino sector.

In the charged scalars sector, besides the charged would-be Goldstone boson, we have two charged scalar fields (we have already omitted the charged would-be Goldstone boson)

[TABLE]

with the non-zero eigenvalues given by

[TABLE]

Notice that

[TABLE]

Notice that $\mu^{2}_{d}>0$ does not disappear in the mass difference above because, in oder to reproduce the Klein-Gordon equation for each component, a real scalar has a $1/2$ factor in the mass term related to the mass of a complex scalar.

IV Lepton masses and the PMNS matrix

In Sec. II we have seen that at tree level the neutrinos are massless, the $a_{6}$ term in Eq. (5) induce diagrams like those in Fig. 1 and it is possible to implement the mechanism of Ref. Ma:2006km for radiative generation of neutrinos mass. In fact, the diagram in Fig. 1 are exactly calculable from the exchange of Re$D^{0}_{1,2}$ and Im$D^{0}_{1,2}$

[TABLE]

where $a=1,2;k=s,d$; $m_{Ra}$ and $m_{Ia}$ are the masses of $\textrm{Re}D^{0}_{1,2}$ and $\textrm{Im}D^{0}_{1,2}$, respectively. In Eq. (18) $Y_{ik}Y_{jk}$ corresponds to $G^{\nu}_{is}G^{\nu}_{js}$ when the coupling is with the $N_{s}$; and $Y_{ik}Y_{jk}$ corresponds to $G^{\nu}_{id}G^{\nu}_{jd}$ when the coupling is with the $N_{d}$, and finally $M_{s}$ is the mass of the right-handed neutrino $N_{s}$, and $M_{d}$ is the common mass of the right-handed neutrino in the doublet of $S_{3}$, $N_{d}$ i.e, $M_{2}=M_{3}\equiv M_{d}$. We can define $\Delta^{2}_{a}=m^{2}_{Ra}-~{}m^{2}_{Ia}=~{}4a_{6}v^{2}_{SM}$, and $m^{2}_{0a}=(m^{2}_{Ra}+m^{2}_{aI})/2$, $a=1,2$.

If $\Delta^{2}\ll m^{2}_{a0}$ we obtain

[TABLE]

where $M_{s}$ is the mass of the right-handed neutrino $N_{s}$ and $M_{d}$ is the common mass of the neutrinos $N_{d}$. Under the condition in which the scalars are mass degenerated i.e., $b_{3}=0$ in (17) we obtain just a factor 2 in Eq.(19). Below, for simplicity, we will consider the case $b_{3}=0$.

In order to obtain the active neutrinos masses we assume a normal hierarchy and, without loss of generality, that $M_{s}\sim M_{d}$ and will be represented from now on by $M_{R}$. $M^{\nu}$ is diagonalized with a unitary matrix $V_{L}^{\nu}$ i.e., $\hat{M}^{\nu}~{}=~{}V^{\nu T}_{L}M^{\nu}V^{\nu}_{L}$, where $\hat{M}^{\nu}=diag(m_{1},m_{2},m_{3})\approx(0,\sqrt{\delta m^{2}_{12}},\sqrt{\delta m^{2}_{23}})$. Taken the central values in PDG we have $\hat{M}^{\nu}\approx(0,x,y)$.

In the charged lepton sector we assume their masses at the central values in PDG $\hat{M^{l}}=(0.510,105.658,1776.86)$ GeV. It is important to note from these considerations, that there exist a multitude of other possibilities which satisfy also the masses squared differences and the astrophysical limits in the active neutrino sector. Each one corresponds to different parameterization of the unitary matrices $V^{l}_{L,R},V^{\nu}_{L}$.

We will obtain the neutrinos masses from Eq. (19). We have as free parameters $a_{6}$, $M_{R}$ and the Yukawas [$m_{+1}=m_{+2}\equiv m_{+}$ are also still free but they will enter only in the leptonic decays considered in Sec. VI]. In the Fig. 2 we show the dependence of $a_{6}$ with respect to the main Yukawas $G_{\tau d}^{\nu}$ in (a) and $G_{es}^{\nu}$ in (b) for fixed $M_{R}$ values. Notice that $G_{\tau d}^{\nu}$ are essentially of the same order of magnitude, while the rest $G_{ed,\mu d,\mu s,\tau s}^{\nu}$ are suppressed by four orders of magnitude when comparing with any specific value of those $G_{\tau d,es}^{\nu}$.

The mass matrices in the charged lepton sector $M^{l}$ are diagonalized by a bi-unitary transformation $\hat{M}^{l}=V^{l\dagger}_{L}M^{l}V^{l}_{R}$ and $\hat{M}^{l}=diag(m_{e},m_{\mu},m_{\tau})$. The relation between symmetry eigenstates (primed) and mass (unprimed) fields are $l^{\prime}_{L,R}=V^{l}_{L,R}l_{L,R}$ and $\nu^{\prime}_{L}=V^{\nu}_{L}\nu_{L}$, where $l^{\prime}_{L,R}=(e^{\prime},\mu^{\prime},\tau^{\prime})^{T}_{L,R}$, $l_{L,R}=(e,\mu,\tau)^{T}_{L,R}$, $\nu^{\prime}_{L}=(\nu_{e},\nu_{\mu},\nu_{\tau})^{T}_{L}$ and $\nu_{L}=(\nu_{1},\nu_{2},\nu_{3})_{L}$. Defining the lepton mixing matrix as $V_{PMNS}=V^{l\dagger}_{L}V^{\nu}_{L}$, it means that this matrix appears in the charged currents coupled to $W_{\mu}^{+}$. We have tested the robustness of our fitting of the lepton masses and the leptonic mixing matrix by using several parametrization corresponding to the values of the Yukawa couplings given in Table 2. We omit the respective matrices $V^{l}_{L}$ and $V^{\nu}_{L}$ but in all cases we have obtained:

[TABLE]

which is in agreement within the experimental error data at 3$\sigma$ given by GonzalezGarcia:2012sz

[TABLE]

and we see that it is possible to accommodate all lepton masses and the PMNS matrix. Here we do not consider $CP$ violation.

V Dark Matter

As we said before, the present model may have a multi-component DM spectrum, which means that many particles may contribute to the relic density of DM, but we will consider the simplest example where one of the $C\!P$ even scalar, say $R_{1}$, and one of the $C\!P$ odd scalar, say $I_{1}$, as the dark matter candidates, each case is considered separately for simplicity. A two inert doublet model without right-handed neutrinos and scalar singlet was considered in Ref. Fortes:2014dca .

As usual, in order to determine the relic density, we solve the Boltzmann equation. Firstly, considering $R$ as the candidate, we have

[TABLE]

where $\langle\sigma|v|\rangle$ is the annihilation cross section already thermally averaged and $H$ is the Hubble constant. In the thermal equilibrium, the number density of DM Beltran:2008xg is

[TABLE]

where $g=1$ for a scalar DM. When solving the Boltzmann equation we obtain the equation for the relic density:

[TABLE]

where $M_{Pl}=1.22\times 10^{19}$ GeV is the Planck mass, $x_{F}=m_{R}/T_{F}$, where $m_{R}$ is the mass of the neutral scalar and $T_{F}$ is the temperature at freeze-out, the terms $a$ and $b$ result from the partial wave expansion of $\sigma|v|=a+bv^{2}$. The number of relativistic degrees of freedom $g_{*}=118.375$ is a result of the SM particles plus three right-handed neutrinos, five neutral scalars, two pseudo-scalars and two charged scalars. The evaluation of $x_{F}$ leads to

[TABLE]

where the unitary parameter $\widetilde{c}\approx 5/4$.

Here we will consider the solution for the relic density which, at the same time, solves the charged lepton masses and neutrino masses given in Eq. (18) for the sets of parameters showed in Tables 3 and 4, in order to obtain the PMNS matrix. We call them scenario 1 and 2, when $R_{1}$ and $I_{1}$ is the DM candidate, respectively. In both scenarios the Yukawas values adjust the squared masses differences for the neutrinos and the PMNS.

With the numbers in Tables 3 and 4 we obtain the mixing matrix, that is:

[TABLE]

and the charged leptons has the following Yukawas: $G^{l}_{11}=0.0004219$, $G^{l}_{12}=0.000515$, $G^{l}_{22}=0.00374772$, $G^{l}_{13}=-0.000801115$, $G^{l}_{23}=-0.0035675$, $G^{l}_{33}=0.00367627$, and we obtain $m_{e}=0.510$ MeV, $m_{\mu}=105.658$ MeV and $m_{\tau}=1776.86$ MeV, and the mixing matrix is

[TABLE]

As previously defined the mixing matrix for the leptonic sector $V_{PMNS}=V^{l\dagger}_{L}V^{\nu}_{L}$, From Eqs. (26) and (27) we obtain again Eq.(20).

To perform DM calculation we have used MicrOmegas package micromegas . For instance, let us consider scenario 1, where $R_{1}$ is the DM candidate. In the range of parameters used by us, DM annihilates mainly in $W^{+}W^{-}$. Once again we emphasize that other solutions in other annihilation channels do exist. We have chosen the following parameters for the couplings, and vacuum expected value: $G_{\tau d}=2.81\times 10^{-7}$, $G_{\tau s}=1\times 10^{-11}$, $v_{\zeta}\lesssim\Lambda=1000$ GeV, $\lambda^{\prime}=2.7\times 10^{-2}$, $\lambda^{\prime\prime}=0.34$, $\mu^{2}_{d}=2.809\,(\textrm{TeV})^{2}$, for values of other parameters see Table 3. With this parameters choice $m_{R1}=85.15$ GeV. The parameter dependence of $m_{R1}$ is presented in the last sections. So, the dominant contributions for $\Omega$ are 99% in $R_{1}R_{1}\rightarrow W^{+}W^{-}$. In this case, $x_{F}\sim 23.8$. The annihilation cross section is $\langle\sigma v\rangle=1.0\times 10^{-26}$ cm${}^{3}/$s and the DM-nucleus cross section for spin-independent elastic scattering is numerically given by $\sigma_{SI}^{p}=5.84\times 10^{-46}$ cm2 and $\sigma_{SI}^{n}=6.70\times 10^{-46}$ cm2.

In scenario 2 we consider $I_{1}$ as the DM candidate. In this case, as a result of the parameter choice ($G_{\tau d}=3.75\times 10^{-7}$, $G_{\tau s}=1\times 10^{-11}$, $v_{\zeta}\lesssim\Lambda=1000$ GeV, $\lambda^{\prime}=1$, $\lambda^{\prime\prime}=0.12$, $\mu^{2}_{d}=2830.24$ TeV2), $I_{1}$ annihilates 97% in $I_{1}I_{1}\rightarrow hh$ and 2% in $I_{1}I_{1}\rightarrow b\overline{b}$. The value of other parameters can be seen in Table 3. The annihilation cross section is $\langle\sigma v\rangle=4.55\times 10^{-28}$ cm${}^{3}/$s and the DM-nucleus cross section for spin-independent elastic scattering is numerically given by $\sigma_{SI}^{p}=6.40\times 10^{-45}$ cm2 and $\sigma_{SI}^{n}=7.35\times 10^{-45}$ cm2.

In these two scenarios, we had set $R_{1}$ and $I_{1}$ as DM candidates, making them lighter than the others possible neutral scalars. We emphasize that other choices for DM are possible so that other annihilation channels may also give interesting signatures. There is also the possibility that two, three or even four of the neutral scalars contribute partially to the DM density, but this case is beyond the scope of this paper.

In Fig. 3 we present the fluxes of photons, positrons and antiprotons in the scenario 1 with $m_{DM}=85.15$ GeV and the scenario 2 with $m_{DM}=113.70$ GeV, where the upper limits of the energy spectrum are determined by the DM masses since annihilation occurs near at rest. The model can accommodate DM candidtes with smaller masses than the values above.

VI The leptonic decays $l_{i}\to l_{j}\gamma$ and $l_{i}\to l_{j}l_{k}\bar{l}_{k}$.

In this section we study the impact of the new particles, the charged scalars $D_{1,2}^{+}$ and the right-handed neutrinos $N_{s,2,3}$ in the lepton flavor violating processes $l_{i}\to l_{j}\gamma$. Here we will consider these rare decays in two cases: one in which we do not care with DM solutions and one in which we use the parameters for having a DM candidates that also give the correct lepton masses and the PMNS.

In terms of the leptons mass eigenstates, the interactions with charged scalars from Eq. (II) are written as

[TABLE]

$i,k=e,\mu,\tau$, the values of the entries of the matrices $G_{i,d}^{\nu}$ and $G_{i,s}^{\nu}$ are given in Table 3.

In the model, the allowed lepton flavor violation (LFV) decays $l_{i}\to l_{j}\gamma$ and $l_{i}\to l_{j}l_{k}\bar{l}_{k}$ arise only at the 1-loop level. These diagrams are generated by the known SM contribution $W~{}\&~{}\nu_{l}$ and by the new content $D_{1}^{+}\&N_{2}$, $D_{2}^{+}\&N_{3}$, $D_{1}^{+}\&N_{s}$ and $D_{2}^{+}\&N_{s}$.

For $l_{i}\to l_{j}\gamma$, it is known that the SM contribution is extraordinarily suppressed with respect to the experimental capabilities of detection, see Table 5. As we will show below, the new particle content in the model predicts signals close to the experimental upper limits for the space of our considered allowed parameters. Regarding the three body decay $l_{i}\to l_{j}l_{k}\bar{l}_{k}$, it arises when in $l_{i}\to l_{j}\gamma$ we attach to the photon the $\gamma l\bar{l}$ coupling. In the following we are interested in presenting the $\mu\to ee\bar{e}$ channel, because it provides interesting results near the experimental upper limit, while all the other channels are out of the experimental interest region because the devices are unable of reaching such suppressed signals.

In our study we have solved the amplitudes and the loop integrals with the help of Mathematica, FeynCalc  Mertig:1990an ; Shtabovenko:2016sxi , and Package-X Patel:2015tea .

VI.1 Predictions of $\mu\to e\gamma$ and $\mu\to ee\bar{e}$ in the scotogenic model without dark matter.

We start our numerical analysis of the decays in the scotogenic model without DM content. Accordingly to the Yukawa values derived in the Sec. IV (see Fig. 2), the obtained values of the Yukawas are $G_{ad,as}^{\nu}\in[10^{-11},10^{-1}]$, they satisfy the neutrino masses. Notice that $G_{\tau d}^{\nu}\simeq G_{es}^{\nu}\gg G_{ed,\mu d,\mu s,\tau s}^{\nu}$. We recall that all the previous analyse were done in the case of $b_{3}=0$ in which the charged scalars are mass degenerated. As starting point we consider the mass of the charged scalar in the range $m_{+}\in$ [80,750] GeV, $N_{2,3,s}$ degenerated as well with values $m_{N}\in$ [250, 4000] GeV. We have tested the five different parameterizations of the $V_{L}^{l}$ matrices derived in the Sec. IV (see Table 2). They can be separated into two sets which will have two different behaviours in the processes, the set A is conformed by the parameterizations P1 and P5, and the B by P2, P3 and P4.

For the channel $\mu\to e\gamma$ such situation occurs when $G_{\tau d}^{\nu}=G_{es}^{\nu}\sim 10^{-1}$ and $G_{ed,\mu d,\mu s,\tau s}^{\nu}\sim 10^{-5}$. In the Fig. 5 (a)-(b) it is shown the Br$(\mu\to e\gamma)$ as function of the sterile neutrino mass $m_{N}\in[250,4000]$ GeV with given values for the charged scalar $m_{D^{+}}=80,250,500,750$ GeV. The current experimental upper limit Br$(\mu\to e\gamma)^{\text{Exp}}<4.2\times 10^{-13}$, indicated with the red line in the plots, will constrain the right-handed neutrino mass $m_{N}$ for given values of $m_{+}$ in order to respect such limit, those constraints are listed in the Table 6, where it is evident that the $V_{L}^{l}$ parameterization A in Fig. 5 (a) allows a lighter mass for the right-handed neutrino than the parameterization B in Fig. 5 (b).

Regarding to the subcase $\mu\to ee\bar{e}$, we are able to predict the branching ratio from the neutrino right-handed mass constraints obtained for the $\mu\to e\gamma$ channel. These predictions are organized in the Table 7, and such values are indicated with the green line in the Fig. 5 (c) and (d), being of the same order of magnitude $\sim 10^{-15}$ for both scenarios.

About the analogous tau decays, for the same space of parameter values than in the $\mu\to e\gamma$ case, and respecting the obtained mass bounds, we have found that Br$(\tau\to e\gamma)\leq 10^{-12}$ and Br$(\tau\to e\gamma)\leq 10^{-14}$, which are beyond the current and upcoming experimental capabilities of detection, see Table 5 for comparison.

VI.2 Predictions of $l_{i}\to l_{j}\gamma$ in the scotogenic model with dark matter

As commented in the Sec. V, the model can be extended to include DM. In order to estimate the consequences on the transition $\mu\to e\gamma$, we consider the Yukawa values given in Table 3. The resulting prediction with $G_{\tau d}^{\nu}=G_{es}^{\nu}=10^{-7}$ is

[TABLE]

which is beyond the scope of detection.

VII Conclusions

Here we have considered an extension of the SM with three scalar doublets of $SU(2)$ with $S_{3}$ and $\mathbb{Z}_{2}$ symmetries. We had analysed all the mass spectra in the scalar sectors and used the scotogenic mechanism for generating neutrino masses. Moreover, we had obtained the PMNS matrix once the unitary matrices which diagonalize the lepton masses are obtained. Although the model can have many DM candidates, we have shown two cases in which the DM candidate is a $C\!P$ even scalar (scenario-1) and other one in which the DM is composite of $C\!P$ odd scalar (scenario-2). But we emphasize that other possible choices for DM candidates are possible, considering for example, smaller masses, since besides the SM-like scalar, we have eight additional neutral scalars in the model. The study of other candidates and other channels of annihilation will be done soon. We had exemplified in some range of parameters space, two DM candidates for the model. For the scenarios 1 and 2 presented, DM annihilates mainly in $W^{+}W^{-}$ and $hh$ respectively. We have also presented some fluxes for this model. Of course, there may be other possible scenarios which could explain the Galactic gamma ray excess, as well as the the PAMELA and AMS-02 results. These processes could tightly constrain the parameter space of this sort of scotogenic models.

The considered scotogenic model without DM provides optimistic predictions for possible detection of the LFV decay $\mu\to e\gamma$ due to our solution space of the Yukawa values, which adjusts the squared masses differences for the neutrinos and the PMNS matrix. Our estimations predict a mass for the right-handed neutrino starting from $m_{N}>295$ GeV, and from $\mu\to e\gamma$ we predict Br$(\mu\to ee\bar{e})\lesssim 10^{-15}$. On the other hand, considering DM content in the model we found that Br$(\mu\to e\gamma)\sim 10^{-34}$, which is out of detection range.

Acknowledgements.

ACBM thanks CAPES for financial support, JM thanks to FAPESP for financial support under the processe number 2013/09173-5, and VP thanks to CNPq for partial financial support.