Neutral Higgs decays $H \rightarrow Z \gamma,\gamma\gamma$ in 3-3-1 models

TL;DR

This paper investigates how 3-3-1 models affect neutral Higgs boson decays into gamma gamma and Z gamma, highlighting the impact of new charged particles and model parameters on decay signals and deviations from Standard Model predictions.

Contribution

It provides a detailed analysis of loop-induced Higgs decays in 3-3-1 models, including the effects of charged Higgs and gauge bosons, and explores the sensitivity of decay rates to model parameters.

Findings

Large $v_3$ suppresses contributions from heavy particles, tightening bounds on decay signals.

Lower $v_3$ allows significant contributions from heavy charged particles, leading to possible destructive or constructive interference.

The branching ratio of a new neutral Higgs $h^0_3$ into gamma gamma is highly sensitive to the parameter $eta$.

Abstract

The significance of new physics appearing in the loop-induced decays of neutral Higgs bosons into pairs of dibosons $\gamma\gamma$ and $Z\gamma$ will be discussed in the framework of the 3-3-1 models based on a recent work~\cite{Okada:2016whh}, where the Higgs sector becomes effectively the same as that in the two Higgs doublet models (2HDM) after the first symmetry breaking from $SU(3)_L$ scale into the electroweak scale. For large $SU(3)_L$ scale $v_3\simeq10$ TeV, dominant one-loop contributions to the two decay amplitudes arise from only the single charged Higgs boson predicted by the 2HDM, leading to that experimental constraint on the signal strength $\mu^{331}_{\gamma\gamma}$ of the Standard Model-like Higgs boson decay $h\rightarrow \gamma\gamma$ will result in a strict upper bound on the signal strength $\mu^{331}_{Z\gamma}$ of the decay $h\rightarrow\, Z\gamma$. For a particular model with lower $v_3$ around 3 TeV, contributions from heavy charged gauge and Higgs bosons may have the same order, therefore may give strong destructive or constructive correlations. As a by-product, a deviation from the SM prediction $|\mu^{331}_{\gamma\gamma}-1| \le 0.04$ still allows $|\mu^{331}_{Z\gamma}-1|$ to reach values near 0.1. We also show that there exists an $CP$-even neutral Higgs boson $h^0_3$ predicted by the 3-3-1 models, but beyond the 2HDM, has an interesting property that the branching ratio Br$(h^0_3\rightarrow \gamma\gamma)$ is very sensitive to the parameter $\beta$ used to distinguish different 3-3-1 models.

Tables

Table 1. Table 1: Feynman rules for the SM-like Higgs boson couplings with charged Higgs bosons

Vertex	Coupling: $-i{\lambda }_{hss}$
$-i{\lambda }_{h{H}^{+}{H}^{-}}$	$iv\left[2s_{12}{c}_{12}\left(-{\lambda }_1{c}_{12}{c}_{\alpha }+{\lambda }_2{s}_{12}{s}_{\alpha }\right)+\left(s_{\alpha }{c}_{12}^3-c_{\alpha }{s}_{12}^3\right){\lambda }_{12}-c_{\delta }{\tilde{\lambda }}_{12}\right]$
$-i{\lambda }_{h{H}^A{H}^{-A}}$	$i{c}_{13}^2\left\{v\left[s_{\alpha }{c}_{12}\left({\lambda }_{12}+t_{13}^2{\lambda }_{23}\right)-c_{\alpha }{s}_{12}\left(2{\lambda }_1+t_{13}^2\left({\lambda }_{13}+{\tilde{\lambda }}_{13}\right)\right)\right]+v_3{t}_{13}\left(\frac{2f{s}_{\alpha }}{v_3}-c_{\alpha }{\tilde{\lambda }}_{13}\right)\right\}$
$-i{\lambda }_{h{H}^B{H}^{-B}}$	$i{c}_{23}^2\left\{v\left[s_{\alpha }{c}_{12}\left(2{\lambda }_2+t_{23}^2\left({\lambda }_{23}+{\tilde{\lambda }}_{23}\right)\right)-c_{\alpha }{s}_{12}\left({\lambda }_{12}+t_{23}^2{\lambda }_{13}\right)\right]+v_3{t}_{23}\left(s_{\alpha }{\tilde{\lambda }}_{23}-\frac{2f{c}_{\alpha }}{v_3}\right)\right\}$

Table 2. Table 2: Yukawa couplings of the SM-like Higgs boson

	$-i{Y}_{h\overline{e_a}{e}_{a}L,R}$	$-i{Y}_{h\overline{u_i}{u}_{i}L,R}$	$-i{Y}_{h\overline{u_3}{u}_{3}L,R}$	$-i{Y}_{h\overline{d_i}{d}_{i}L,R}$	$-i{Y}_{h\overline{d_3}{d}_{3}L,R}$
	$-i\frac{m_{e_a}}{v}\left(c_{\delta }-\frac{s_{\delta }}{t_{12}}\right)$	$-i\frac{m_{u_i}}{v}\left(c_{\delta }-\frac{s_{\delta }}{t_{12}}\right)$	$-i\frac{m_{u_3}}{v}\left(c_{\delta }+t_{12}{s}_{\delta }\right)$	$-i\frac{m_{d_i}}{v}\left(c_{\delta }+t_{12}{s}_{\delta }\right)$	$-i\frac{m_{d_3}}{v}\left(c_{\delta }-\frac{s_{\delta }}{t_{12}}\right)$

Table 3. Table 3: Feynman rules for couplings of the SM-like Higgs boson to Higgs and gauge bosons.

Vertex	Coupling:	Vertex	Coupling
$g_{h{W}^{+}{W}^{-}}$	$g{m}_W{c}_{\delta }$	$g_{h{Y}^{+A}{Y}^{-A}}$	$g{m}_W{c}_{\alpha }{s}_{12}$
$g_{h{V}^{+B}{V}^{-B}}$	$-g{m}_W{s}_{\alpha }{c}_{12}$	$g_{h{H}^{-}{W}^{+}}$	$\frac{g{s}_{\delta }}{2}$
$g_{h{H}^{-A}{Y}^A}$	$-\frac{g{c}_{13}{c}_{\alpha }}{2}$	$g_{h{H}^{-B}{V}^B}$	$\frac{g{c}_{23}{s}_{\alpha }}{2}$

Table 4. Table 4: Feynman rules of couplings with Z 𝑍 Z to charged Higgs and gauge bosons.

Vertex	Coupling
$g_{Z{H}^{+}{H}^{-}}$	$\frac{g}{2c_W}\left(c_{\theta }{c}_{2W}+\frac{s_{\theta }\left[\sqrt{3}{c}_W^2\left(1-2s_{12}^2\right)+3\beta s_W^2\right]}{3c_W\sqrt{1-{\beta }^2{t}_W^2}}\right)$
$g_{Z{H}^A{H}^{-A}}$	$\frac{g}{2c_W}\left(c_{\theta }\left[s_{13}^2-\left(1+\sqrt{3}\beta \right)s_W^2\right]+\frac{s_{\theta }\left[\sqrt{3}{c}_W^2\left(s_{13}^2-2\right)+3\beta \left(\sqrt{3}\beta +c_{13}^2\right)s_W^2\right]}{3c_W\sqrt{1-{\beta }^2{t}_W^2}}\right)$
$g_{Z{H}^B{H}^{-B}}$	$\frac{ig}{2c_W}\left(-c_{\theta }\left[s_{23}^2+\left(\sqrt{3}\beta -1\right)s_W^2\right]+\frac{s_{\theta }\left[\sqrt{3}{c}_W^2\left(s_{23}^2-2\right)+3\beta \left(\sqrt{3}\beta -c_{23}^2\right)s_W^2\right]}{3c_W\sqrt{1-{\beta }^2{t}_W^2}}\right)$
$g_{Z{W}^{+}{H}^{-}}$	$-\frac{g{m}_W\left(2s_{12}{c}_{12}{s}_{\theta }\right)}{\sqrt{3\left(1-{\beta }^2{t}_W^2\right)}}$
$g_{Z{Y}^A{H}^{-A}}$,	$\frac{g^2{c}_{13}}{4}\{\frac{}{}{c}_{\theta }{c}_W[s_{12}(1+(2+\sqrt{3}\beta )t_W^2)v+t_{13}(1-\sqrt{3}\beta t_W^2)v_3]$
$g_{Z{Y}^{-A}{H}^A}$	$+\frac{s_{\theta }}{3\sqrt{1-{\beta }^2{t}_W^2}}[s_{12}(\sqrt{3}-3\beta (2+\sqrt{3}\beta )t_W^2)v+\sqrt{3}{t}_{13}(1+3{\beta }^2{t}_W^2)v_3]\}$
$g_{Z{V}^B{H}^{-B}}$,	$\frac{g^2{c}_{23}}{4}\{\frac{}{}{c}_{\theta }{c}_W[c_{12}(-1+(-2+\sqrt{3}\beta )t_W^2)v-t_{23}(1+\sqrt{3}\beta t_W^2)v_3]$
$g_{Z{V}^{-B}{H}^B}$	$+\frac{s_{\theta }}{3\sqrt{1-{\beta }^2{t}_W^2}}[c_{12}(\sqrt{3}-3\beta (-2+\sqrt{3}\beta )t_W^2)v+\sqrt{3}{t}_{23}(1+3{\beta }^2{t}_W^2)v_3]\}$

Table 5. Table 5: Couplings of Z 𝑍 Z with fermions

$f$	$g_L^f$	$g_R^f$
$e_a$	$-\frac{1}{2}+s_W^2+\frac{t_{\theta }{c}_W\left(1-\sqrt{3}\beta t_W^2\right)}{2\sqrt{3\left(1-{\beta }^2{t}_W^2\right)}}$	$s_W^2\left(1-\frac{t_{\theta }\beta }{c_W\sqrt{1-{\beta }^2{t}_W^2}}\right)$
$u_i$	$\frac{1}{2}-\frac{2}{3}{s}_W^2+\frac{t_{\theta }{c}_W\left(\beta t_W^2-\sqrt{3}\right)}{6\sqrt{1-{\beta }^2{t}_W^2}}$	$-\frac{2}{3}{s}_W^2\left(1-\frac{t_{\theta }\beta }{c_W\sqrt{1-{\beta }^2{t}_W^2}}\right)$
$u_3$	$\frac{1}{2}-\frac{2}{3}{s}_W^2+\frac{t_{\theta }{c}_W\left(\beta t_W^2+\sqrt{3}\right)}{6\sqrt{1-{\beta }^2{t}_W^2}}$	$-\frac{2}{3}{s}_W^2\left(1-\frac{t_{\theta }\beta }{c_W\sqrt{1-{\beta }^2{t}_W^2}}\right)$
$d_i$	$-\frac{1}{2}+\frac{1}{3}{s}_W^2+\frac{t_{\theta }{c}_W\left(\beta t_W^2-\sqrt{3}\right)}{6\sqrt{1-{\beta }^2{t}_W^2}}$	$\frac{1}{3}{s}_W^2\left(1-\frac{t_{\theta }\beta }{c_W\sqrt{1-{\beta }^2{t}_W^2}}\right)$
$d_3$	$-\frac{1}{2}+\frac{1}{3}{s}_W^2+\frac{t_{\theta }{c}_W\left(\beta t_W^2+\sqrt{3}\right)}{6\sqrt{1-{\beta }^2{t}_W^2}}$	$\frac{1}{3}{s}_W^2\left(1-\frac{t_{\theta }\beta }{c_W\sqrt{1-{\beta }^2{t}_W^2}}\right)$

Table 6. Table 6: Feynman rules for triple gauge couplings relating with the decay h → Z ​ γ , γ ​ γ → ℎ 𝑍 𝛾 𝛾 𝛾 h\rightarrow Z\gamma,\gamma\gamma .

Vertex	Coupling
$-i{g}_{Z{W}^{+\nu }{W}^{-\lambda }}$	$-ig{c}_W{c}_{\theta }$
$-i{g}_{Z{Y}^A{Y}^{-A}}$	$\frac{ig}{2}\left[c_{\theta }\left(-c_W+\sqrt{3}\beta s_W{t}_W\right)+s_{\theta }\sqrt{3-3{\beta }^2{t}_W^2}\right]$
$-i{g}_{Z{V}^B{Y}^{-B}}$	$\frac{ig}{2}\left[c_{\theta }\left(c_W+\sqrt{3}\beta s_W{t}_W\right)+s_{\theta }\sqrt{3-3{\beta }^2{t}_W^2}\right]$

Table 7. Table 7: Branching ratios of the SM Higgs boson decays with mass of 125.09 GeV.

$b\overline{b}$	${\tau }^{+}{\tau }^{-}$	${\mu }^{+}{\mu }^{-}$	$c\overline{c}$	$gg$	$\gamma \gamma$	$Z\gamma$	$WW$	$ZZ$	${\Gamma }_h^{\mathrm{SM}}$ (GeV)
0.5809	0.06256	$2.171\times {10}^{-4}$	$0.02884$	$0.0818$	$0.00227$	$0.001541$	$0.2152$	$0.02641$	$4.10\times {10}^{-3}$

Table 8. Table 8: Numerical contributions of S ​ U ​ ( 3 ) L 𝑆 𝑈 subscript 3 𝐿 SU(3)_{L} particles to F 21 331 subscript superscript 𝐹 331 21 F^{331}_{21} and F γ ​ γ 331 subscript superscript 𝐹 331 𝛾 𝛾 F^{331}_{\gamma\gamma} , see Eqs. ( 81 ) and  ( 83 ), where F 21 , s ​ v 331 ≡ F 21 , s ​ v ​ v 331 + F 21 , v ​ s ​ s 331 subscript superscript 𝐹 331 21 𝑠 𝑣 subscript superscript 𝐹 331 21 𝑠 𝑣 𝑣 subscript superscript 𝐹 331 21 𝑣 𝑠 𝑠 F^{331}_{21,sv}\equiv F^{331}_{21,svv}+F^{331}_{21,vss} .

$\beta$	$s_{\delta }$	$t_{12}$	$\frac{F_{21,s}^{331}}{\mathrm{Re}\left[F_{21}^{\mathrm{SM}}\right]}$	$\frac{F_{21,v}^{331}}{\mathrm{Re}\left[F_{21}^{\mathrm{SM}}\right]}$	$\frac{F_{21,sv}^{331}}{\mathrm{Re}\left[F_{21}^{\mathrm{SM}}\right]}$	$\frac{F_{\gamma \gamma ,s}^{331}}{\mathrm{Re}\left[F_{\gamma \gamma }^{\mathrm{SM}}\right]}$	$\frac{F_{\gamma \gamma ,v}^{331}}{\mathrm{Re}\left[F_{\gamma \gamma }^{\mathrm{SM}}\right]}$	$\delta {\mu }_{Z\gamma }$	$\delta {\mu }_{\gamma \gamma }$
$\frac{2}{\sqrt{3}}$	$2\times {10}^{-2}$	$1.5$	$-3.3\times {10}^{-4}$	$3\times {10}^{-5}$	$-1.6\times {10}^{-4}$	$-6\times {10}^{-4}$	$5.5\times {10}^{-4}$	$4.4$	$6.5$
$\frac{2}{\sqrt{3}}$	$-2\times {10}^{-2}$	$1.5$	$\sim {10}^{-6}$	$3\times {10}^{-5}$	$-1.5\times {10}^{-4}$	$\sim {10}^{-6}$	$5.3\times {10}^{-4}$	$-5.4$	$-6$
$\frac{2}{\sqrt{3}}$	$2\times {10}^{-2}$	$0.5$	$1.3\times {10}^{-4}$	$-9\times {10}^{-5}$	$-5\times {10}^{-5}$	$2.3\times {10}^{-4}$	$2.2\times {10}^{-4}$	$6.8$	$8.1$
$\frac{2}{\sqrt{3}}$	$-2\times {10}^{-2}$	$0.5$	$-4.2\times {10}^{-4}$	$-9\times {10}^{-5}$	$-4\times {10}^{-5}$	$-7.5\times {10}^{-4}$	$2.1\times {10}^{-4}$	$-7.5$	$-7.4$
$\frac{2}{\sqrt{3}}$	$-{10}^{-3}$	$1.5$	$-1.6\times {10}^{-4}$	$3\times {10}^{-5}$	$-1.6\times {10}^{-4}$	$-2.9\times {10}^{-4}$	$5.4\times {10}^{-4}$	$-0.8$	$-0.2$

Table 9. Table 9: Numerical contributions of S ​ U ​ ( 3 ) L 𝑆 𝑈 subscript 3 𝐿 SU(3)_{L} particles to F 21 331 subscript superscript 𝐹 331 21 F^{331}_{21} and F γ ​ γ 331 subscript superscript 𝐹 331 𝛾 𝛾 F^{331}_{\gamma\gamma} for large λ ~ 12 = 5 subscript ~ 𝜆 12 5 \tilde{\lambda}_{12}=5 and small m h 2 0 = 600 subscript 𝑚 subscript superscript ℎ 0 2 600 m_{h^{0}_{2}}=600 GeV.

$\beta$	$s_{\delta }$	$t_{12}$	$\frac{F_{21,s}^{331}}{\mathrm{Re}\left[F_{21}^{\mathrm{SM}}\right]}$	$\frac{F_{21,v}^{331}}{\mathrm{Re}\left[F_{21}^{\mathrm{SM}}\right]}$	$\frac{F_{21,sv}^{331}}{\mathrm{Re}\left[F_{21}^{\mathrm{SM}}\right]}$	$\frac{F_{\gamma \gamma ,s}^{331}}{\mathrm{Re}\left[F_{\gamma \gamma }^{\mathrm{SM}}\right]}$	$\frac{F_{\gamma \gamma ,v}^{331}}{\mathrm{Re}\left[F_{\gamma \gamma }^{\mathrm{SM}}\right]}$	$\delta {\mu }_{Z\gamma }$	$\delta {\mu }_{\gamma \gamma }$
$\frac{2}{\sqrt{3}}$	${10}^{-3}$	$1.7$	$-1.46\times {10}^{-2}$	$4\times {10}^{-5}$	$-1.7\times {10}^{-4}$	$-2.64\times {10}^{-2}$	$5.7\times {10}^{-4}$	$-3.1$	$-4.7$
$\frac{2}{\sqrt{3}}$	$-{10}^{-3}$	$1.7$	$-1.44\times {10}^{-2}$	$4\times {10}^{-5}$	$-1.7\times {10}^{-4}$	$-2.61\times {10}^{-2}$	$5.7\times {10}^{-4}$	$-3.6$	$-5.3$
$\frac{2}{\sqrt{3}}$	$3\times {10}^{-2}$	$1.5$	$-1.24\times {10}^{-2}$	$3\times {10}^{-5}$	$-1.6\times {10}^{-4}$	$-2.23\times {10}^{-2}$	$5.5\times {10}^{-4}$	$4.4$	$5.2$
$\frac{2}{\sqrt{3}}$	$-3\times {10}^{-2}$	$1.5$	$-9.6\times {10}^{-3}$	$3\times {10}^{-5}$	$-1.5\times {10}^{-4}$	$-1.75\times {10}^{-3}$	$5.3\times {10}^{-4}$	$-9.6$	$-12.3$

Table 10. Table 10: For model in Ref.  Coriano:2018coq , numerical contributions of S ​ U ​ ( 3 ) L 𝑆 𝑈 subscript 3 𝐿 SU(3)_{L} particles to F 21 331 subscript superscript 𝐹 331 21 F^{331}_{21} and F γ ​ γ 331 subscript superscript 𝐹 331 𝛾 𝛾 F^{331}_{\gamma\gamma} . Notations are given from caption of table  8 .

$\beta$	$s_{\delta }$	$t_{12}$	$\frac{F_{21,s}^{331}}{\mathrm{Re}\left[F_{21}^{\mathrm{SM}}\right]}$	$\frac{F_{21,v}^{331}}{\mathrm{Re}\left[F_{21}^{\mathrm{SM}}\right]}$	$\frac{F_{21,sv}^{331}}{\mathrm{Re}\left[F_{21}^{\mathrm{SM}}\right]}$	$\frac{F_{\gamma \gamma ,s}^{331}}{\mathrm{Re}\left[F_{\gamma \gamma }^{\mathrm{SM}}\right]}$	$\frac{F_{\gamma \gamma ,v}^{331}}{\mathrm{Re}\left[F_{\gamma \gamma }^{\mathrm{SM}}\right]}$	$\delta {\mu }_{Z\gamma }$	$\delta {\mu }_{\gamma \gamma }$
$\sqrt{3}$	${10}^{-3}$	$1.5$	$-1.8\times {10}^{-4}$	$-1.6\times {10}^{-3}$	$-4\times {10}^{-3}$	$-3.2\times {10}^{-4}$	$2.2\times {10}^{-2}$	$-1.6$	$4.8$
$\sqrt{3}$	$-{10}^{-3}$	$1.5$	$-1.6\times {10}^{-4}$	$-1.7\times {10}^{-3}$	$-4\times {10}^{-3}$	$-2.9\times {10}^{-4}$	$2.2\times {10}^{-2}$	$-2$	$4.2$

Table 11. Table 11: For model in Ref.  Coriano:2018coq , numerical contributions of S ​ U ​ ( 3 ) L 𝑆 𝑈 subscript 3 𝐿 SU(3)_{L} particles to F 21 331 subscript superscript 𝐹 331 21 F^{331}_{21} and F γ ​ γ 331 subscript superscript 𝐹 331 𝛾 𝛾 F^{331}_{\gamma\gamma} with m h 2 0 = 800 subscript 𝑚 subscript superscript ℎ 0 2 800 m_{h^{0}_{2}}=800 GeV. Notations are given from caption of table  8 .

${\lambda }_1$	${\tilde{\lambda }}_{12}$	$s_{\delta }$	$t_{12}$	$\frac{F_{21,s}^{331}}{\mathrm{Re}\left[F_{21}^{\mathrm{SM}}\right]}$	$\frac{F_{21,v}^{331}}{\mathrm{Re}\left[F_{21}^{\mathrm{SM}}\right]}$	$\frac{F_{21,sv}^{331}}{\mathrm{Re}\left[F_{21}^{\mathrm{SM}}\right]}$	$\frac{F_{\gamma \gamma ,s}^{331}}{\mathrm{Re}\left[F_{\gamma \gamma }^{\mathrm{SM}}\right]}$	$\frac{F_{\gamma \gamma ,v}^{331}}{\mathrm{Re}\left[F_{\gamma \gamma }^{\mathrm{SM}}\right]}$	$\delta {\mu }_{Z\gamma }$	$\delta {\mu }_{\gamma \gamma }$
$1.95$	$8$	${10}^{-3}$	$1.5$	$-1.22\times {10}^{-2}$	$-1.7\times {10}^{-3}$	$-4.4\times {10}^{-3}$	$-2.21\times {10}^{-2}$	$2.2\times {10}^{-2}$	$-4.$	$0.4$
$1.95$	$8$	$-{10}^{-3}$	$1.5$	$-1.21\times {10}^{-2}$	$-1.7\times {10}^{-3}$	$-4.4\times {10}^{-3}$	$-2.19\times {10}^{-2}$	$2.2\times {10}^{-2}$	$-4.5$	$-0.17$
$1.$	$5$	$-2\times {10}^{-2}$	$1.95$	$-1.6\times {10}^{-3}$	$-9.7\times {10}^{-4}$	$-5\times {10}^{-3}$	$-1.2\times {10}^{-2}$	$2.4\times {10}^{-2}$	$-7.7$	$-4$

Table 12. Table 12: Numerical contributions of S ​ U ​ ( 3 ) L 𝑆 𝑈 subscript 3 𝐿 SU(3)_{L} particles to F 21 331 subscript superscript 𝐹 331 21 F^{331}_{21} and F γ ​ γ 331 subscript superscript 𝐹 331 𝛾 𝛾 F^{331}_{\gamma\gamma} . Numerical fixed values of unknown parameters are: β = 2 / 3 𝛽 2 3 \beta=2/\sqrt{3} , t 12 = 0.1 subscript 𝑡 12 0.1 t_{12}=0.1 , λ ~ 12 = − 1 subscript ~ 𝜆 12 1 \tilde{\lambda}_{12}=-1 .

$\beta$	$m_{h_2^0}$ [TeV]	$s_{\delta }$	$\frac{F_{21,s}^{331}}{F_{21}^{\mathrm{SM}}}$	$\frac{F_{21,v}^{331}}{F_{21}^{\mathrm{SM}}}$	$\frac{F_{21,sv}^{331}}{F_{21}^{\mathrm{SM}}}$	$\frac{F_{\gamma \gamma ,s}^{331}}{F_{\gamma \gamma }^{\mathrm{SM}}}$	$\frac{F_{\gamma \gamma ,v}^{331}}{F_{\gamma \gamma }^{\mathrm{SM}}}$	$\delta {\mu }_{Z\gamma }$	$\delta {\mu }_{\gamma \gamma }$
$\frac{2}{\sqrt{3}}$	$1$	$-{10}^{-3}$	$3.4\times {10}^{-4}$	$-1.4\times {10}^{-4}$	$\simeq 0$	$6.1\times {10}^{-3}$	$9\times {10}^{-5}$	$-1.8$	$-1.2$
$\frac{2}{\sqrt{3}}$	$0.6$	$-{10}^{-3}$	$1.2\times {10}^{-3}$	$-1.4\times {10}^{-4}$	$\simeq 0$	$-2.1\times {10}^{-3}$	$9\times {10}^{-5}$	$-1.7$	$-0.9$
$\frac{2}{\sqrt{3}}$	$1$	$-2\times {10}^{-2}$	$-1.4\times {10}^{-3}$	$-1.4\times {10}^{-4}$	$\simeq 0$	$-2.6\times {10}^{-3}$	$9\times {10}^{-5}$	$-23.9$	$-23.9$
$\frac{2}{\sqrt{3}}$	$0.6$	$-2\times {10}^{-2}$	$6.6\times {10}^{-4}$	$-1.2\times {10}^{-3}$	$\simeq 0$	$-1.2\times {10}^{-3}$	$9\times {10}^{-5}$	$-23.7$	$-23.6$

Table 13. Table 13: Triple Higgs couplings of h 3 0 subscript superscript ℎ 0 3 h^{0}_{3} involving to the decay h 3 0 → γ ​ γ , Z ​ γ → subscript superscript ℎ 0 3 𝛾 𝛾 𝑍 𝛾 h^{0}_{3}\rightarrow\gamma\gamma,Z\gamma

Vertex	Coupling: $-i{\lambda }_{S_i{S}_j{S}_k}$
$h_3^0{H}^{+}{H}^{-}$	$-i\left[\left(1+s_{12}^2\right){\lambda }_{13}+s_{12}^2{\lambda }_{23}\right]v_3$
$h_3^0{H}^A{H}^{-A}$	$-i\left[2s_{13}^2{\lambda }_3+c_{13}^2{\lambda }_{13}+{\tilde{\lambda }}_{13}\right]v_3$
$h_3^0{H}^B{H}^{-B}$	$-i\left[2s_{23}^2{\lambda }_3+c_{23}^2{\lambda }_{23}+{\tilde{\lambda }}_{23}\right]v_3$

Table 14. Table 14: Heavy neutral Higgs boson couplings to charged Higgs and gauge bosons.

Vertex	Coupling	Vertex	Coupling
$g_{h_2^0{W}^{+}{W}^{-}}$	$g{m}_W{s}_{\delta }$
$g_{h_2^0{Y}^{+A}{Y}^{-A}}$	$g{m}_W{s}_{12}{s}_{\alpha }$	$g_{h_3^0{Y}^{+A}{Y}^{-A}}$	$\frac{g^2{v}_3}{2}$
$g_{h_2^0{V}^{+B}{V}^{-B}}$	$g{m}_W{c}_{12}{c}_{\alpha }$	$g_{h_3^0{V}^{+B}{V}^{-B}}$	$\frac{g^2{v}_3}{2}$
$g_{h_2^{0}HW}$	$\frac{g{c}_{\delta }}{2}$	$g_{h_3^{0}HW}$	0
$g_{h_2^0{H}^{-A}{Y}^A}$	$-\frac{g{c}_{13}{s}_{\alpha }}{2}$	$g_{h_3^0{H}^{-A}{Y}^A}$	$\frac{g{s}_{13}}{2}$
$g_{h_2^0{H}^{-B}{Y}^B}$	$-\frac{g{c}_{23}{c}_{\alpha }}{2}$	$g_{h_3^0{H}^{-B}{Y}^B}$	$\frac{g{s}_{23}}{2}$

Table 15. Table 15: h i 0 ​ Z ​ Z subscript superscript ℎ 0 𝑖 𝑍 𝑍 h^{0}_{i}ZZ couplings in the limit s θ 2 = 0 , c θ 2 = 1 formulae-sequence subscript superscript 𝑠 2 𝜃 0 subscript superscript 𝑐 2 𝜃 1 s^{2}_{\theta}=0,\,c^{2}_{\theta}=1 .

Vertex	coupling $g_{h_i^{0}ZZ}$
$hZZ$	$\frac{g{m}_W}{c_W^2}\left[c_{\delta }\left(1+\frac{2\sqrt{3}{s}_{\theta }{c}_{\theta }{c}_W\left(1-2s_{12}^2-\sqrt{3}{t}_W^2\beta \right)}{3\sqrt{1-{\beta }^2{t}_W^2}}\right)-\frac{4s_{\delta }{c}_W{s}_{\theta }{c}_{\theta }{s}_{12}{c}_{12}}{\sqrt{3\left(1-{\beta }^2{t}_W^2\right)}}\right]$
$h_2^{0}ZZ$	$\frac{g{m}_W}{c_W^2}\left[s_{\delta }\left(1+\frac{2\sqrt{3}{s}_{\theta }{c}_{\theta }{c}_W\left(1-2s_{12}^2-\sqrt{3}{t}_W^2\beta \right)}{3\sqrt{1-{\beta }^2{t}_W^2}}\right)+\frac{4c_{\delta }{c}_W{s}_{\theta }{c}_{\theta }{s}_{12}{c}_{12}}{\sqrt{3\left(1-{\beta }^2{t}_W^2\right)}}\right]$

Table 16. Table 16: Couplings of Z 𝑍 Z with exotic fermions

$F$	$g_L^F$	$g_R^F$
$E_a$	$g_R^{E_a}-\frac{t_{\theta }{c}_W}{\sqrt{3\left(1-{\beta }^2{t}_W^2\right)}}$	$-\frac{\left(-1+\sqrt{3}\beta \right)s_W^2}{2}\left(1-\frac{t_{\theta }\beta }{c_W\sqrt{1-{\beta }^2{t}_W^2}}\right)$
$J_i$	$g_R^{J_i}+\frac{t_{\theta }{c}_W}{\sqrt{3\left(1-{\beta }^2{t}_W^2\right)}}$	$\frac{\left(-1+3\sqrt{3}\beta \right)s_W^2}{6}\left(1-\frac{t_{\theta }\beta }{c_W\sqrt{1-{\beta }^2{t}_W^2}}\right)$
$J_3$	$g_R^{J_3}-\frac{t_{\theta }{c}_W}{\sqrt{3\left(1-{\beta }^2{t}_W^2\right)}}$	$-\frac{\left(1+3\sqrt{3}\beta \right)s_W^2}{6}\left(1-\frac{t_{\theta }\beta }{c_W\sqrt{1-{\beta }^2{t}_W^2}}\right)$

Equations

$$Q=T_3+\beta T_8+X,$$

$$\displaystyle L^{\prime}_{aL}=\left(\begin{array}[]{c}e^{\prime}_{a}\\ -\nu^{\prime}_{a}\\ E^{\prime}_{a}\\ \end{array}\right)_{L}\sim\left(1~{},3^{*}~{},-\frac{1}{2}+\frac{\beta}{2\sqrt{3}}\right),\hskip 14.22636pta=1,2,3,$$

$$e_{aR}^{\prime }\sim \left(1,1,-1\right),{\nu }_{aR}^{\prime }\sim \left(1,1,0\right),E_{aR}^{\prime }\sim \left(1,1,-\frac{1}{2}+\frac{\sqrt{3}\beta }{2}\right),$$

$$Q_{iL}^{\prime }$$

$$u_{aR}^{\prime }$$

$$J_{iR}^{\prime }$$

$$\chi$$

$$\eta$$

$${\chi }^0$$

$${\mathcal{L}}_{\mathrm{lepton}}^Y$$

$${\mathcal{L}}_{\mathrm{quark}}^Y$$

$$-Y_{ij}^J{\overline{Q^{\prime }}}_{iL}\chi J_{jR}^{\prime }-Y_{33}^J{\overline{Q^{\prime }}}_{3L}{\chi }^{\ast }{J}_{3R}^{\prime }+h.c.,$$

$$m_{e_a}$$

$$D_{\mu }\equiv {\partial }_{\mu }-ig{T}^a{W}_{\mu }^a-i{g}_{X}X{T}^9{X}_{\mu },$$

$$\displaystyle W^{a}_{\mu}T^{a}=\frac{1}{2}\left(\begin{array}[]{ccc}W^{3}_{\mu}+\frac{1}{\sqrt{3}}W^{8}_{\mu}&\sqrt{2}W^{+}_{\mu}&\sqrt{2}Y^{+A}_{\mu}\\ \sqrt{2}W^{-}_{\mu}&-W^{3}_{\mu}+\frac{1}{\sqrt{3}}W^{8}_{\mu}&\sqrt{2}V^{+B}_{\mu}\\ \sqrt{2}Y^{-A}_{\mu}&\sqrt{2}V^{-B}_{\mu}&-\frac{2}{\sqrt{3}}W^{8}_{\mu}\\ \end{array}\right),$$

$$W_{\mu }^{\pm }=\frac{1}{\sqrt{2}}\left(W_{\mu }^1\mp i{W}_{\mu }^2\right),Y_{\mu }^{\pm A}=\frac{1}{\sqrt{2}}\left(W_{\mu }^4\mp i{W}_{\mu }^5\right),V_{\mu }^{\pm B}=\frac{1}{\sqrt{2}}\left(W_{\mu }^6\mp i{W}_{\mu }^7\right),$$

$$A=\frac{1}{2}+\beta \frac{\sqrt{3}}{2},B=-\frac{1}{2}+\beta \frac{\sqrt{3}}{2}.$$

$$g_2=g,g_1=g_X\frac{g}{\sqrt{6g^2+{\beta }^2{g}_X^2}},$$

$$\frac{g_X^2}{g^2}=\frac{6s_W^2}{1-(1+{\beta }^2)s_W^2}=\frac{6s_W^2}{c_W^2(1-{\beta }^2{t}_W^2)},$$

$$m_Y^2\equiv m_{Y^{\pm A}}^2=\frac{g^2}{4}(v_3^2+v_1^2),m_V^2\equiv m_{V^{\pm B}}^2=\frac{g^2}{4}(v_3^2+v_2^2),m_W^2\equiv m_{W^{\pm }}^2=\frac{g^2}{4}(v_1^2+v_2^2).$$

$$s_{ij}\equiv$$

$$s_{331}\equiv \sin {\theta }_{331}$$

$$\left(\begin{array}{c}{X}_{\mu }\\ W_{\mu }^3\\ W_{\mu }^8\end{array}\right)$$

$$C$$

$$s_{\theta }\equiv \sin \theta$$

$$V_h$$

$$\displaystyle\left(\begin{array}[]{c}\phi^{\pm}_{W}\\ H^{\pm}\\ \end{array}\right)$$

$$\displaystyle\left(\begin{array}[]{c}\phi^{\pm A}_{Y}\\ H^{\pm A}\\ \end{array}\right)$$

$$\displaystyle\left(\begin{array}[]{c}\phi^{\pm B}_{V}\\ H^{\pm B}\\ \end{array}\right)$$

$$R(x)\equiv\left(\begin{array}[]{cc}c_{x}&-s_{x}\\ s_{x}&c_{x}\\ \end{array}\right).$$

$$f={\lambda }_{13}{t}_{12}{v}_3=\frac{{\lambda }_{23}{v}_3}{t_{12}}.$$

$$M_r^2=\left(\begin{array}{ccc}2{\lambda }_1{s}_{12}^2{v}^2+{\lambda }_{13}{v}_3^2& t_{12}\left({\lambda }_{12}{c}_{12}^2{v}^2-{\lambda }_{13}{v}_3^2\right)& 0\\ t_{12}\left({\lambda }_{12}{c}_{12}^2{v}^2-{\lambda }_{13}{v}_3^2\right)& 2c_{12}^2{\lambda }_2{v}^2+t_{12}^2{\lambda }_{13}{v}_3^2& 0\\ 0& 0& s_{12}^2{\lambda }_{13}{v}^2+2{\lambda }_3{v}_3^2\end{array}\right).$$

$$R(\alpha )M_r^{\prime 2}{R}^T(\alpha )=\mathrm{diag}(m_{h_1^0}^2,m_{h_2^0}^2),$$

$$\alpha$$

$$\tan 2\delta$$

Full text

I Introduction

One of the most important channels confirming the existence of the Standard Model-like (SM-like) Higgs boson is the loop-induced decay channel $h\rightarrow\gamma\gamma$. Experimentally, the respective signal strength $\mu_{\gamma\gamma}\equiv(\sigma.\mathrm{Br})_{\mathrm{obs}}/(\sigma.\mathrm{Br})_{\mathrm{SM}}$, which is the observed product of the Higgs boson production cross section ($\sigma$) and its branching ratio (Br) in units of the corresponding values predicted by the standard model (SM) Tanabashi:2018oca , has been updated recently by ATLAS and CMS Aaboud:2018ezd ; Sirunyan:2018ouh ; Aaboud:2018xdt . There is another loop-induced decay $h\rightarrow\,Z\gamma$, which the branching ratio (Br) predicted by the SM is $\mathrm{Br}(h\rightarrow\,Z\gamma)=1.54\times 10^{-3}$ $(\pm 5.7\%)$ corresponding to the Higgs boson mass $m_{h}=125.09$ GeV Heinemeyer:2013tqa ; deFlorian:2016spz . This decay channel has not been observed experimentally. The recent upper constraints of the signal strength are $\mu_{Z\gamma}<6.6$ and $\mu_{Z\gamma}<3.9$ from ATLAS and CMS  Aaboud:2017uhw ; Sirunyan:2018tbk , respectively. In the future project from LHC with its High Luminosity (HL-LHC) and High Energy (HE-LHC), precision measurements for the signal strengths of the two decays $h\rightarrow Z\gamma$ and $h\rightarrow\gamma\gamma$ can reach the respective values of $\mu_{Z\gamma}=1\pm 0.23$ and $\mu_{\gamma\gamma}=1\pm 0.04$ for both ATLAS and CMS Cepeda:2019klc . In addition, the ATLAS expected significance to the $h\rightarrow\,Z\gamma$ channel is hoped to be $4.9~{}\sigma$ with $3000~{}\mathrm{fb}^{-1}$.

In theoretical side, the loop-induced decays of the SM-like Higgs boson mentioned above are important for searching as well as constraining new physics predicted by recent SM extensions, constructed to explain various current experimental data beyond the SM predictions. In the SM, leading contributions to the amplitudes of both decays $h\rightarrow\gamma\gamma,Z\gamma$ are at the one-loop level and relate with $W$ and fermion mediation. On the other hand, SM extensions usually contain new charged particles including scalar, fermions, and gauge bosons spin 1. If any of them couple with the SM-like Higgs boson, they will contribute to the decay amplitude $h\rightarrow\gamma\gamma$ from the one-loop level. Normally, these particles also couple with the SM gauge boson $Z$, hence give one-loop contributions to the decay amplitude $h\rightarrow\,Z\gamma$ too. It seems that the Br of the two decays $h\rightarrow\gamma\gamma,Z\gamma$ have certain relations so that the recent experimental constraint of $\mu_{\gamma\gamma}$ may result in a respective constraint on $\mu_{Z\gamma}$.

The theoretical studies of loop effects caused by new particles on the SM-like Higgs decays including $h\rightarrow\gamma\gamma,Z\gamma$ have been done recently in many SM extensions such as 2HDM Fontes:2014xva ; Kanemura:2018yai ; Bhattacharyya:2013rya ; Bhattacharyya:2014oka , where a thorough investigation in Ref. Fontes:2014xva concerned strong correlations between two signal strengths $\mu_{\gamma\gamma,Z\gamma}$. Hence, the experimental data of $\mu_{\gamma\gamma}$ can be used as an efficient way to predict theoretically constraints on the $\mu_{Z\gamma}$. In left-right models, the decay channel $h\rightarrow\gamma\gamma$ can be used to constrain new heavy charged gauge boson masses Bandyopadhyay:2019jzq . On the other hand, it seems that the result one-loop contribution to the decay amplitude $h\rightarrow Z\gamma$ Martinez:1989kr ; Maiezza:2016bzp has not been discussed for further studying this decay properties using the latest experimental data of the SM-like Higgs boson such as the mass and decay $h\rightarrow\gamma\gamma$. In a recent scotogenic model, new singly and doubly charged Higgs bosons contribute to both loop-induced decay amplitudes $h\rightarrow\gamma\gamma,Z\gamma$ Chen:2019okl . But in this framework, the recent experimental data of the decay $h\rightarrow\gamma\gamma$ predicts a very small a very small deviation from the SM $|\mu_{Z\gamma}-1|<4\%$. In Higgs triplet models Blunier:2016peh , the situation is the same where it was pointed out that Br$(h\rightarrow Z\gamma)$ is usually smaller than Br$(h\rightarrow\gamma\gamma)$. Tiny values of $|\mu_{Z\gamma}-1|$ have been shown recently in other Higgs extensions of the SM Kanemura:2018esc .

In this work, we will focus on another class of the SM extensions, called the 3-3-1 models, which are constructed from the gauge group $\mathbf{SU(3)_{C}}\times SU(3)_{L}\times U(1)_{X}$ Singer:1980sw ; Valle:1983dk ; Pisano:1991ee ; Frampton:1992wt ; Foot:1992rh ; Foot:1994ym . These models have many interesting features which cannot be explained in the SM framework, for example they can give explanations of the existence of three fermion families, the electric charge quantization deSousaPires:1998jc , the sources of CP violations Montero:1998yw ; Montero:2005yb , the strong $CP$-problem Pal:1994ba ; Dias:2002gg ; Dias:2003zt ; Dias:2003iq . In general, one of the most important parameters to distinguish different 3-3-1 models is denoted as $\beta$, which defines electric charges of new particles through the following electric charge operator,

[TABLE]

where $T_{3}$ and $T_{8}$ are two diagonal generators of the $SU(3)$ group, $X$ is the $U(1)$ charge. Apart from the popular 3-3-1 models with values of $\beta=\pm\frac{1}{\sqrt{3}},\pm\sqrt{3}$, other models with $\beta=0,\pm\frac{2}{\sqrt{3}},\;\frac{1}{3\sqrt{3}}$ have been discussed phenomenologically Hue:2015mna ; Buras:2016dxz ; RamirezBarreto:2019bpx . Different phenomenological aspects in models with arbitrary $\beta$ were also discussed CarcamoHernandez:2005ka ; Cao:2016uur ; Yue:2013qba ; Buras:2014yna ; Buras:2013dea ; Martinez:2014lta ; Hue:2017lak ; Long:2018fud . As we will see, the model contains nine electroweak gauge bosons, four of them are identified as the SM-like particles. The remaining include one heavy neutral gauge boson $Z^{\prime}$ and the two pairs of heavy gauge bosons with electric charges depending on Eq. (1), see a detailed pedagogical calculation in Ref. Buras:2012dp . As usual, all particles get masses from three Higgs $SU(3)_{L}$ triplets, including a neutral $CP$-even Higgs component with a large expectation vacuum value (vev) $v_{3}$ that generates masses for heavy $SU(3)_{L}$ particles. The three Higgs triplets also contain new charged Higgs bosons that may contribute to the amplitudes of the loop-induced decays of neutral Higgs bosons, including the SM-like one. Correlations among these Higgs and gauge contributions will predict the allowed regions of the parameter space satisfying the current experimental data of $h\rightarrow\gamma\gamma$. It is interesting to estimate how large of the allowed values of $\mu_{Z\gamma}$ can be.

The decay $h\rightarrow\gamma\gamma$ was mentioned in some particular 3-3-1 models for constraining the parameter space Caetano:2013nya ; CarcamoHernandez:2019vih . Both $h\rightarrow\gamma\gamma,Z\gamma$ were also mentioned previously in the 3-3-1 models Cao:2016uur ; Yue:2013qba , but some nontrivial contributions to the amplitude of the decay $h\rightarrow\,Z\gamma$ were not included. In this work, we will study effects of heavy particles predicted by the 3-3-1 models on the two signal strengths of the two decays of the SM-like Higgs bosons $h\rightarrow\gamma\gamma,Z\gamma$, using more general analytic formulas of one-loop contributions to the decay amplitude $h\rightarrow\,Z\gamma$ introduced recently Degrande:2017naf ; Hue:2017cph . For simplicity in calculating the physical states of the neutral $CP$-even Higgs bosons, the Higgs potential of the 3-3-1 models will be considered as an effective 2HDM after the first breaking step $SU(3)_{L}\times U(1)_{X}\rightarrow SU(2)_{L}\times U(1)_{Y}$. This form of the Higgs potential was mentioned in detail in Ref. Okada:2016whh for studying a 3-3-1 model with $\beta=-1/\sqrt{3}$. This property of the 3-3-1 models was mentioned previously Martinez:2014lta . The Higgs potential in this limit can be applied to a general 3-3-1 model keeping $\beta$ as a free parameter ($331\beta$). This can be seen by the fact that the model contains two $SU(3)_{L}$ Higgs triplets having components the same as those appear in the 2HDM. The physical states of neutral Higgs bosons then can be determined exactly at the tree level. Recent theoretical constraints on the Higgs sector of 2HDM Haisch:2018djm can be used to constrain the allowed regions of the parameter space relating with those included in the $331\beta$.

On the other hand, the 331$\beta$ contains another heavy neutral Higgs boson $h^{0}_{3}$ that does not couple with SM particles, except the SM-like Higgs boson. Hence, if it is the lightest among those beyond the SM particles, its main decay channels are the tree level decay into a pair of SM-like Higgs boson and loop-induced decays to pairs of gluons and gauge bosons $\gamma\gamma,Z\gamma$. An investigation to determine which decay channels can be used to distinguish different 3-3-1 models will also be presented.

Our work is arranged as follows. Section II summarizes contents of the 3-3-1 models investigated in this work. All couplings and analytic formulas needed for calculating the Brs and signal strengths of the $h,h^{0}_{3}\rightarrow\gamma\gamma,Z\gamma$ are presented in Section III. Numerical results are shown in Section IV. Important remarks and inclusions are pointed out in Section V. Finally, there are three appendices listing more detailed calculations on couplings, particular analytic formulas for one-loop contributions to the decay amplitudes $h,h^{0}_{3}\rightarrow\gamma\gamma,\,Z\gamma$, and interesting numerical illustrations.

II 3-3-1 model with arbitrary $\beta$

II.1 The model review

In this section, we summarize the particle content of the model $331\beta$. Left- and right-handed leptons are assigned to $SU(3)_{L}$ antitriplets and singlets, respectively:

[TABLE]

where in the parentheses present the representations and the hypercharge $X$ of the gauge groups $SU(3)_{C}$, $SU(3)_{L}$ and $U(1)_{X}$, respectively. The model includes three right handed (RH) neutrinos $\nu^{\prime}_{aR}$ and heavy exotic leptons $E^{\prime a}_{L,R}$.

The quark sector is arranged to guarantee anomaly cancellation, namely

[TABLE]

where $i=1,2$, $a=1,2,3$, and $J_{aL,R}$ are exotic quarks predicted by the 331$\beta$ model. There is another arrangement that the model contains three left-handed lepton $SU(3)_{L}$ triplets, one quark $SU(3)_{L}$ triplet and two other quark $SU(3)_{L}$ antitriplets. But, it was shown that the two arrangements are equivalent in the sense that they predict the same physics Descotes-Genon:2017ptp ; Hue:2018dqf .

To generate masses for gauge bosons and fermions, three scalar triplets are introduced as follows

[TABLE]

where $A,B$ denote electric charges defined in Eq. (1): $A=\frac{1+\beta\sqrt{2}}{2}$ and $B=\frac{-1+\beta\sqrt{2}}{2}$. These Higgs bosons develop vevs defined as $\langle\chi^{0}\rangle=\frac{v_{3}}{\sqrt{2}},\,\langle\rho^{0}\rangle=\frac{v_{2}}{\sqrt{2}},\,\langle\eta^{0}\rangle=\frac{v_{1}}{\sqrt{2}}$, leading to

[TABLE]

The symmetry breaking happens in two steps: $SU(3)_{L}\otimes U(1)_{X}\xrightarrow{v_{3}}SU(2)_{L}\otimes U(1)_{Y}\xrightarrow{v_{1},v_{2}}U(1)_{Q}$. It is therefore reasonable to assume that $v_{3}>v_{1},v_{2}$. At the second breaking step, $\rho$ and $\eta$ play roles of the two $SU(2)_{L}$ doublets similar to 2HDM, except differences in coupling with fermions. Masses and physical states of all particles are summarized as follows.

II.2 Fermions

Masses and physical states of the fermions are derived from the following Yukawa Lagrangian

[TABLE]

We note that depending on particular values of $\beta$, additional Yukawa terms may appear but a $Z_{2}$ symmetry can be imposed to exclude them, see an example given in Ref. Okada:2016whh .

As mentioned above, the SM-like fermions get masses from their couplings to two Higgs triplets $\eta$ and $\rho$, similarly to the 2HDM. On the other hand, the up (down) quarks couple to both Higgs triplets, leading to a different feature from four popular types of 2HDM, where all up (down) quarks couple to the same Higgs doublet in order to avoid tree level flavor changing neutral currents (FCNCs), see for example in Ref. Craig:2012vn . As a result, many interesting properties relating with the SM-like fermion couplings were pointed out to distinguish 3-3-1 models and 2HDMs Okada:2016whh .

The exotic fermions couple to only the Higgs triplet $\chi$. Accordingly, the neutral Higgs sector in Ref. Okada:2016whh has a property that the $\chi^{0}$ does not contribute to the SM-like Higgs boson, it therefore decouples with all exotic fermions. Hence, they do not contribute to the one-loop decay amplitudes $h\rightarrow\gamma\gamma,Z\gamma$.

The SM-like fermion masses are determined based on discussions in refs. Okada:2016whh ; Buras:2012dp ; Hue:2017lak , where the mixing between quarks are safely ignored in this work. Then all fermion mass matrices are diagonal. Correspondingly, the original fermion states are physical, hence they will be denoted by $e_{aL,R},u_{aL,R}$ and $d_{aL,R}$. The fermion masses are given as follows:

[TABLE]

where $Y^{f}_{ab}=0~{}\forall a\neq b$, $f=e,u,d,F$ and $F=J,E$. The relations (29) will be used to determine Feynman rules of Yukawa couplings in Lagrangians (27) and (28).

II.3 Gauge bosons

The model contains nine electroweak (EW) gauge bosons corresponding to the 9 generators of the EW gauge group $SU(3)_{L}\otimes U(1)_{X}$. The covariant derivative is defined as 222This definition is different from Ref. Okada:2016whh by $T^{9}$. Buras:2012dp ; Hue:2017lak ; Cao:2016uur ,

[TABLE]

where $T^{9}=1/\sqrt{6}$, $g$ and $g_{X}$ are coupling constants of the two groups $SU(3)_{L}$ and $U(1)_{X}$, respectively. The matrix $W^{a}T^{a}$, where $T^{a}=\lambda_{a}/2$ corresponding to a triplet representation, is

[TABLE]

where we have defined the mass eigenstates of the nondiagonal gauge bosons as

[TABLE]

and $A,B$ are electric charges of the corresponding gauge bosons calculated based on Eq. (1),

[TABLE]

We note that $B$ is also the electric charge of the new leptons $E_{a}$.

The symmetry breaking happens in two steps: $SU(3)_{L}\otimes U(1)_{X}\xrightarrow{v_{3}}SU(2)_{L}\otimes U(1)_{Y}\xrightarrow{v_{1},v_{2}}U(1)_{Q}$, corresponding to the following transformation of the neutral gauge bosons form the original basis to the final physical one: $X_{\mu},\,W^{3}_{\mu},\,W^{8}_{\mu}\xrightarrow{\theta_{331}}\,B_{\mu},\,W^{3}_{\mu},\,Z^{\prime}_{\mu}\xrightarrow{\theta_{W}}A_{\mu},\,Z_{\mu},\,Z^{\prime}_{\mu}\xrightarrow{\theta}A_{\mu},\,Z_{1\mu},\,Z_{2\mu}$. After the first step, five gauge bosons will be massive and the remaining four massless gauge bosons can be identified with the before-symmetry-breaking SM gauge bosons. The two physical states $Z_{1,2}$ are mixed from the SM and heavy gauge bosons $Z_{\mu}$ and $Z^{\prime}_{\mu}$.

It is well-known that

[TABLE]

where $g_{2}$ and $g_{1}$ are the two couplings of the of the SM gauge groups $SU(2)_{L}$ and $U(1)_{Y}$, respectively. Using the weak mixing angle defined as $t_{W}=\tan\theta_{W}=g_{1}/g_{2}$ and denoting $s_{W}=\sin\theta_{W}$ and $c_{W}=\cos\theta_{W}$, it is derived that

[TABLE]

which gives a constraint $|\beta|\leq\sqrt{3}$ used in the numerical analysis.

The masses of the gauge bosons given in (35) are

[TABLE]

The matching condition with the SM gives $v^{2}\equiv v_{1}^{2}+v_{2}^{2}\simeq 246\,[\mathrm{GeV}^{2}]$. Based on Refs. Buras:2012dp ; Buras:2014yna , the ratios between vevs are used to define three mixing parameters as follows

[TABLE]

where $i<j$ and $i,j=1,2,3$.

The model predicts three neutral gauge bosons including the massless photon. Defining Buras:2012dp

[TABLE]

the relation between the original and physical base of the neutral gauge bosons are

[TABLE]

where in the limit $v^{2}\ll v_{3}^{2}$, the mixing angle $\theta$ is determined as Buras:2014yna

[TABLE]

and $M^{2}_{Z^{\prime}}=g^{2}v^{2}_{3}/(3s^{2}_{331})+\mathcal{O}(v^{2})$.

To continue, the neutral gauge bosons will be identified as $Z_{1}\equiv Z$ and $Z_{2}\equiv Z^{\prime}$, where $Z$ is the one found experimentally.

II.4 Higgs bosons

The scalar potential is

[TABLE]

The minimum conditions of the Higgs potential can be figured out easily Buras:2012dp ; Hue:2017lak . After that, we can take $\mu_{i}^{2}$ as functions of other independent parameters. These functions are inserted into the Higgs potential (44), which is used to determine the masses and physical states of all Higgs bosons.

The relations between original and mass eigenstates of charged Higgs bosons are Buras:2012dp ; Hue:2017lak :

[TABLE]

where we have define a rotation $R(x)$ as

[TABLE]

The massless states $\phi^{\pm}_{W}$, $\phi^{\pm A}_{Y}$, and $\phi^{\pm\,B}_{V}$ are Goldstone bosons absorbed by the physical gauge bosons.

For neutral Higgs bosons, to avoid the tree level contribution of SM-like Higgs bosons to the flavor changing neutral currents (FCNC) in the quark sector, we follow the aligned limit introduced in Ref. Okada:2016whh , namely

[TABLE]

From this, we will choose $f$ and $\lambda_{23}$ as functions of the remaining, leading to the following form of the squared mass matrix corresponding to the basis $(r_{1},r_{2},r_{3})$:

[TABLE]

As a result, $r_{3}\equiv h^{0}_{3}$ is a physical $CP$-even neutral Higgs boson with mass $m^{2}_{h^{0}_{3}}=\lambda_{13}s^{2}_{12}v^{2}+2\lambda_{3}v_{3}^{2}$. The submatrix $2\times 2$ in Eq. (62) is denoted as $M^{\prime 2}_{r}$. It is diagonalized as follows Okada:2016whh ,

[TABLE]

where

[TABLE]

We also have

[TABLE]

To determine the SM-like Higgs boson, we first look at the Eq. (65), which give $\delta=\mathcal{O}(\frac{v^{2}}{v_{3}^{2}})\simeq 0$ when $v^{2}\ll v_{3}^{2}$. In this limit, $m^{2}_{h}=M^{2}_{11}+v^{2}\times\mathcal{O}(\frac{v^{2}}{v_{3}^{2}})\sim M^{2}_{11}$ while $m^{2}_{h^{0}_{2}}=M^{2}_{22}+v^{2}\times\mathcal{O}(\frac{v^{2}}{v_{3}^{2}})\simeq M^{2}_{22}$. Hence, $h^{0}_{1}\equiv\,h$ is identified with the SM-like Higgs boson found at LHC. Furthermore, in the following calculation we will see more explicitly that the couplings of this Higgs boson are the same as those given in the SM in the limit $\delta\rightarrow 0$.

Because the two mass matrices $M^{\prime 2}_{r}$ given in Eqs. (62) and the one given in Eq. (63) differ from each other by the unitary transformation $R(\alpha)$, their traces are equal, namely Tr$[M^{\prime 2}_{r}]=\mathrm{Tr}[R(\alpha)M^{\prime 2}_{r}R^{T}(\alpha)]=m^{2}_{h^{0}_{1}}+m^{2}_{h^{0}_{2}}$. Accordingly, $\lambda_{13}$ can be written as

[TABLE]

We will choose $\delta$, $m_{h^{0}_{1}}\equiv m_{h}$ and $m_{h^{0}_{2}}$ as input parameters. The $\lambda_{13}$, $\lambda_{12}$ and $\lambda_{2}$ are dependent parameters, namely

[TABLE]

and $\lambda_{13}$ was given in Eq. (73).

The Higgs self-couplings should satisfy all constraints discussed recently to guarantee the vacuum stability of the Higgs potential Sanchez-Vega:2018qje , the perturbative limits, and the positive squared masses of all Higgs bosons. We note that in the case of absence the relations in Eq. (61), the mixing between SM-like Higgs bosons with other heavy neutral Higgs still suppressed due to large $v_{3}>5$ TeV enoungh to cancel the FCNCs in 3-3-1 models Huitu:2019kbm .

III Couplings and analytic formulas involved with loop-induced Higgs decays

III.1 Couplings

From the above discussion on the Higgs potential, we can derive all Higgs self-couplings of the SM-like Higgs boson relating to the decays $h\rightarrow Z\gamma$ and $h\rightarrow\gamma\gamma$, using the interacting Lagrangian $\mathcal{L}_{hHH}=-V_{h}$. The Feynman rules are given in Table 1, where each factor $-i\lambda_{hss}$ corresponds to a vertex $hss$, where $s=H^{\pm},H^{\pm A},H^{\pm B}$.

Based on the Yukawa Lagrangians (27) and (28), the couplings of the SM-like Higgs boson with SM fermions can be determined, see also in table 1, where we have used the relation (64). The notation of the Feynman rule is $-i\left(Y_{h\bar{f}fL}P_{L}+Y_{h\bar{f}fR}P_{R}\right)$ for each vertex $h\bar{f}f$. For simplicity, the Yukawa couplings of the SM-like fermions in this case were identified with those in the SM, as discussed before. Then, we have $Y_{\bar{f}fL}=Y_{\bar{f}fR}$, which are given in table 2.

Both neutral $CP$-even Higgs bosons $h$ and $h^{0}_{2}$ do not couple to exotic fermion in the aligned limit (61). In contrast, $h^{0}_{3}$ couples only to the exotic fermions, while it does not couple to the SM ones.

The couplings of Higgs and gauge bosons are contained in the covariant kinetic terms of the Higgs bosons

[TABLE]

where sums are taken over $s=H^{\pm},H^{\pm A},H^{\pm B}$ and $v=W,Y,V$. In addition, we only list the relevant terms contributing to the decays $h\rightarrow Z\gamma,\gamma\gamma$ and ignore the remaining terms. The Feynman rules for particular couplings in (III.1) are shown in Table 3, where $\partial_{\mu}h\rightarrow-ip_{0\mu}h$ and $\partial_{\mu}s^{\pm Q}\rightarrow-ip_{\pm\mu}s^{\pm Q}$ and the relation  (64) was used. The notations $p_{0}$, $p_{\pm}$ are incoming momenta.

Similarity to the SM-like Higgs boson case,the Feynman rules for the couplings of $Z$ to charged Higgs and gauge bosons in (III.1) are given in table 4.

The couplings of $Z$ and photon $A_{\mu}$ with fermions arise from the covariant kinetic of fermions:

[TABLE]

where $f$ runs over all fermions in the 331$\beta$ model, $Q_{f}$ is the electric charge of the fermion $f$. Values of $g^{f}_{L,R}$ are shown in table 5.

The triple couplings of three gauge bosons arise from the covariant kinetic Lagrangian of the non-Abelian gauge bosons:

[TABLE]

where

[TABLE]

$f^{abc}$ $(a,b,c=1,2,...,8)$ are structure constants of the $SU(3)$ group. They are defined as

[TABLE]

where $\Gamma_{\mu\nu\lambda}(p_{0},p_{+},p_{-})\equiv g_{\mu\nu}(p_{0}-p_{+})_{\lambda}+g_{\nu\lambda}(p_{+}-p_{-})_{\mu}+g_{\lambda\mu}(p_{-}-p_{0})_{\lambda}$, and $v=W,V,Y$. The involved couplings of $Z$ are given in table 6.

These triple couplings were also given in Ref. Diaz:2004fs ; Cao:2016uur in the limit $\theta=0$.

III.2 Partial decay widths and signal strengths of the decays $h\rightarrow Z\gamma,\gamma\gamma$

In the unitary gauge, the above couplings generate one-loop three point Feynman diagrams which contribute to the decay amplitude of the SM-like Higgs boson $h\rightarrow\,Z\gamma$, as given in Fig. 1.

The partial decay width is Gunion:1989we ; Degrande:2017naf

[TABLE]

where the scalar factor $F_{21}$ is determined from one-loop contributions. More general formulas were given in Ref. Hue:2017cph , leading the following expression

[TABLE]

We note that $F^{331}_{21,vss}$ and $F^{331}_{21,svv}$ were not included in previous works Cao:2016uur ; Yue:2013qba .

The detailed analytic formulas of particular notations in (81) are given in appendix B. The partial decay width of the decay $h\rightarrow\gamma\gamma$ can be calculated as Degrande:2017naf ; Hue:2017cph

[TABLE]

where

[TABLE]

see detailed analytic formulas in appendix B. To determine the Br of a SM-like Higgs decay, we need to know the total decay width. In the SM, this quantity is Heinemeyer:2013tqa ; deFlorian:2016spz

[TABLE]

where the partial decay widths are well-known with Higgs boson mass of 125.09 GeV found experimentally Tanabashi:2018oca . The Br of a particular decay channel $h\rightarrow X$, $X=gg,\gamma\gamma,Z\gamma$, is:

[TABLE]

The numerical values are given in table 7 Heinemeyer:2013tqa ; deFlorian:2016spz , where the diphoton decay is consistent with that used in Ref. Aaboud:2018xdt , Br$(h\rightarrow\gamma\gamma)=(2.27\pm 0.07)\times 10^{-3}$.

The recent global signal strength found experimentally by ATLAS is $\mu_{\gamma\gamma}=0.99\pm 0.14$ Aaboud:2018xdt 333This value gives the same numerical discussion with that reported in Khachatryan:2014ira ; Aad:2014eha . .

The total decay width of the SM-like Higgs boson predicted by the $331\beta$ is computed based on the deviations of the Higgs couplings with fermions and gauge bosons between the two models SM and $331\beta$, as given in tables 1 and 3. The result is

[TABLE]

There are three loop-induced decays $h\rightarrow\gamma\gamma,Z\gamma,gg$. The SM-like Higgs boson does not couple with the exotic quarks in the $331\beta$, we can consider only the top quark contribution to the loop contributing to the decay $h\rightarrow gg$. This results in

[TABLE]

where the deviation comes from the $ht\bar{t}$ coupling listed in table 1. This is consistent with recent investigation for $h\rightarrow\gamma\gamma$ in a 3-3-1 model CarcamoHernandez:2019vih .

In the 331$\beta$ framework, the branching ratio of the decay $h\rightarrow\,X$ with $X=\gamma\gamma,Z\gamma$ is

[TABLE]

Many experimental measurements relating to the SM-like Higgs boson were reported in Ref. Khachatryan:2016vau . We consider the SM-like Higgs production through the gluon fusion process $ggF$ at LHC. The respective signal strength predicted by $331\beta$ is defined as:

[TABLE]

where the last value comes from our assumption that only the main contribution from top quark in the loop is considered. The signal strength of an individual loop-induced decay channel is

[TABLE]

The recent signal strengths of the two loop-induced decay $h\rightarrow Z\gamma$ is $\mu_{Z\gamma}<6.6(5.2)$ Aaboud:2017uhw ; Tanabashi:2018oca .

III.3 Decays of the neutral Higgs boson $h^{0}_{3}$

In the above discussion we derived only couplings that contribute to the one-loop amplitudes of the two SM-like Higgs decay channels $h\rightarrow\gamma\gamma,Z\,\gamma$. Other interesting couplings are listed in the appendix A. Here we stress a very interesting property of the heavy neutral Higgs boson $h^{0}_{3}$ that it has only one non-zero coupling with two SM particles, namely only $\lambda_{h^{2}h^{0}_{3}}\neq 0$. We have $m_{h^{0}_{3}}>2m_{h}$ then if $h^{0}_{3}$ is lighter than all other exotic particles predicted by the 331$\beta$ model, only the tree level decay $h^{0}_{3}\rightarrow hh$ appears. Loop-induced decays such as $h^{0}_{3}\rightarrow\,gg,\gamma\gamma,Z\gamma$ also appear, as we will present below. Hence, the total decay width of $h^{0}_{3}$ cannot satisfy the stable condition of a dark matter, $\Gamma_{h^{0}_{3}}<1.3\times 2\pi\times 10^{-42}$ GeV Banerjee:2016vrp ; DeLopeAmigo:2009dc ; Eiteneuer:2017hoh ; Belyaev:2016lok . Anyway, DM candidates as scalar 3-3-1 Higgs bosons were pointed out previously Filippi:2005mt ; Cogollo:2014jia ; deS.Pires:2007gi .

The couplings of neutral heavy Higgs bosons $h^{0}_{2,3}$ to fermions are

[TABLE]

One interesting point is that $h^{0}_{3}$ couples to only exotic fermions, similar to the heavy neutral Higgs appeared in a $SU(2)_{1}\times SU(2)_{2}\times U(1)_{Y}$ model Boucenna:2016qad , where the partial decay width $h^{0}_{3}\rightarrow\,gg$ is  Gunion:1989we ; Spira:1997dg ,

[TABLE]

where $t_{a}\equiv 4m^{2}_{J_{a}}/m^{2}_{h^{0}_{3}}$,

[TABLE]

In the limit $t_{a}\gg 1~{}\forall a=1,2,3$, Eq. (92) can be estimated as Boucenna:2016qad

[TABLE]

Furthermore, the production cross section of this Higgs boson through the gluon-gluon fusion can be estimated from the two gluon decay channel Boucenna:2016qad .

The partial width of the tree level decay $h^{0}_{3}\rightarrow hh$ when $m_{h^{0}_{3}}>2m_{h}$ is Okada:2016whh

[TABLE]

where $\lambda_{13}$ was given in Eq. (73).

The total decay width of the $h^{0}_{3}$ is then

[TABLE]

The last two decays are calculated as follows,

[TABLE]

where

[TABLE]

where $s=H^{\pm},H^{\pm,A},H^{\pm,B}$, $v=Y^{\pm,A},V^{\pm,B}$, and $\{s,v\}=\{H^{\pm,A},Y^{\pm,A}\},\{H^{\pm,B},V^{\pm,B}\}$. The explicit forms in (99) were shown in appendix B.

IV Numerical discussions

IV.1 Signal of the decay $h\rightarrow Z\gamma$ under recent constraints of parameters and the decay $h\rightarrow\gamma\gamma$

In this section, to express quantitative deviations between predictions of the two models 331$\beta$ and the SM for decays $h\rightarrow X$ ($X=\gamma\gamma,Z\gamma$), we define a quantity $\delta\mu_{X}$ as follows

[TABLE]

We also introduce a new quantity $R_{Z\gamma/\gamma\gamma}\equiv|\delta\mu_{Z\gamma}/\delta\mu_{\gamma\gamma}|$ to investigate the relative difference between the two signal strengths, which have many similar properties. The recent allowed values relating with the two photon decay is $-15\%\leq\delta\mu_{\gamma\gamma}\leq 13\%$, corresponding to the recent experimental constraint $\mu_{\gamma\gamma}=0.99\pm 0.14$ Aaboud:2018xdt . The future sensitivities obtained by experiments we accept here are $\mu_{\gamma\gamma}=1\pm 0.04$ and $\mu_{Z\gamma}=1\pm 0.23$ Cepeda:2019klc , i.e. $|\delta\mu_{\gamma\gamma}|\leq 4\%$ and $|\delta\mu_{Z\gamma}|\leq 23\%$, respectively.

Many well-known quantities used in this section are fixed from experiments Tanabashi:2018oca , namely the SM-like Higgs mass $m_{h}=125.09$ GeV; the gauge boson masses $m_{W}$, $m_{Z}$; well-known charged fermion masses; the vev $v\simeq 246$ GeV; and the $SU(2)_{L}$ couplings $g\simeq 0.651$, $\alpha_{\mathrm{em}}=1/137$, $e=\sqrt{4\pi\alpha_{\mathrm{em}}}$, $s^{2}_{W}=0.231$.

The unknown independent parameters used as inputs are $\beta$, $t_{12}$, $SU(3)_{L}$ scale $v_{3}$, the neutral Higgs mixing $s_{\delta}$, the heavy neutral Higgs boson masses $m_{h^{0}_{2}}$, $m_{h^{0}_{3}}$, the triple Higgs self couplings including $\lambda_{1}$, $\tilde{\lambda}_{12}$, $\tilde{\lambda}_{13}$, $\tilde{\lambda}_{23}$, and the exotic fermion masses $m_{E_{a}}$, $m_{J_{a}}$.

The exotic fermion masses $m_{E_{a}}$, $m_{J_{a}}$ affect only the loop-induced decays of $h^{0}_{3}$. We can put $m_{E_{a}}=m_{J_{a}}=m_{F}$ for simplicity. There is a more general case that the mixing between different exotic leptons appear, then the loop with two distinguished fermions will contribute to the $h^{0}_{3}\rightarrow\,Z\gamma$ decay amplitude only.

The $SU(3)_{L}$ scale depends strongly on the heavy neutral gauge boson mass $m_{Z^{\prime}}$, which the lower bound is constrained from experimental searches for decays to pairs of SM leptons $Z^{\prime}\rightarrow\ell\bar{\ell}$, for 3-3-1 models see Coutinho:2013lta , where decays into exotic lepton pairs were included. Accordingly, at LHC@14TeV, $m_{Z^{\prime}}<4$ TeV is excluded at the integrated luminosity of $23~{}\mathrm{fb}^{-1}$ for $\beta=-1/\sqrt{3}$. Recent works have used the $m_{Z^{\prime}}\geq 4$ TeV for models with $\beta=-1/\sqrt{3}$ Ferreira:2019qpf ; Long:2018dun , based on the latest LHC search Aaboud:2017sjh ; Sirunyan:2018exx ; Aad:2019fac . Because $v_{3}\sim\mathcal{O}(1)$ TeV, the $m_{Z^{\prime}}$ is approximately calculated from $m_{Z^{\prime}}^{2}=\frac{g^{2}v_{3}^{2}c^{2}_{W}}{3[1-(1+\beta^{2})s^{2}_{W}]}$. From this, the lower bound of $m_{Z^{\prime}}>4$ TeV corresponds to lower bounds of $v_{3}\geq 10.6,\,10.1,\,8.2,\,3.3$ TeV with respective values of $\beta=0,\pm 1/\sqrt{3},\pm 2/\sqrt{3},\pm\sqrt{3}$. Recent discussion on 3-3-1 models with heavy right-handed neutrinos where $\beta=-1/\sqrt{3}$ and $m_{Z^{\prime}}=3$ TeV is allowed Freitas:2018vnt ; Arcadi:2019uif because the decay of $Z^{\prime}$ into a pair of light exotic neutrinos is included. The respective lower bound of the $SU(3)_{L}$ scale is $v_{3}\geq 7.6$ TeV, which is still the same mentioned bounds. On the other hand, a model with $\beta=\sqrt{3}$ still allows rather low $SU(3)_{L}$ scale, for example $m_{Z^{\prime}}\simeq 3.25$ TeV, corresponding to $v_{3}\simeq 2.7$ TeV Coriano:2018coq . Because the numerical results do not change significantly in the range $7.6\;\mathrm{TeV}<v_{3}<14$ TeV, we will fixed $v=14$ TeV for $|\beta|<\sqrt{3}$ and $v=3$ TeV for $|\beta|=\sqrt{3}$.

The perturbative limits require that the absolute values of all Yukawa and Higgs self-couplings should be less than $\sqrt{4\pi}$ and $4\pi$, respectively. This leads to an upper bound of $t_{12}$ derived from the Yukawa coupling of the top quark in Eq. (29), namely $t_{12}<\sqrt{2\pi}v/m_{t}\simeq 3.5$. Other studies on the 2HDM suggest that $t_{12}>1/60$ Cepeda:2019klc . We will limit that $0.1\leq\,t_{12}\leq 3$, which is consistent with Ref. Okada:2016whh and allows large $|s_{\theta}|\geq 5\times 10^{-3}$.

Considering $m_{h^{0}_{2}},m_{A},m_{H^{\pm}}$, $t_{12}$ and $s_{\delta}$ as parameters of a 2HDM model mentioned in Ref. Kanemura:2018yai , an important constraint can be found as $c_{\delta}>0.99$ for all 2HDMs, leading to rather large range of $|s_{\delta}|<0.14$. But large $s_{\delta}$ prefers that $t_{12}$ is around 1 Aad:2015pla . The recent global fit for 2HDM gives the same result Haller:2018nnx . Lower masses of heavy Higgs bosons are around 1 TeV. As we will show, the recent signal strength of the SM-like Higgs decay $h\rightarrow\gamma\gamma$ gives more strict constraint on $s_{\theta}$, hence we focus on the interesting range $|s_{\theta}|\leq 0.05$. The parameters $\lambda_{2}$ and $\lambda_{12}$ relating with 2HDM affect strongly on $m_{h^{0}_{2}}$. Large $|s_{\theta}|$ results in small allowed values of $m_{h^{0}_{2}}$ in order to keep $\lambda_{12}$ satisfying the perturbative limit. In contrast, all other quantities relating with the $SU(3)_{L}$ symmetry are well allowed. The regions of parameter space chosen here are consistent with the recent works on 2HDM Ferreira:2019iqb ; Kainulainen:2019kyp ; Babu:2018uik . The recent experimental searches for Higgs bosons predicted by 2HDM have been paid much attentions Aaboud:2017gsl . The value of 300 GeV for lower bounds of charged and $CP$-even neutral Higgs bosons are accepted in recent studies on 2HDM Babu:2018uik . Following that, values of $m_{h^{0}_{2}}$ and $m_{H^{\pm}}$ will be chosen to satisfy $m_{h^{0}_{2}},m_{H^{\pm}}\geq 300$ GeV.

We will also consider the case of light charged Higgs masses, which loop contributions to the decay $h\rightarrow\gamma\gamma,Z\gamma$ may be large. Accordingly, the Higgs self-couplings $\tilde{\lambda}_{ij}$ relating with charged Higgs masses in Eqs. (49), (54) and (59), should be negative. Our investigation suggests that $|\tilde{\lambda}_{13,23}|\leq\mathcal{O}(10^{-3})$ while $|\tilde{\lambda}_{12}|$ can be reach order 1. We will consider more detailed in particular numerical investigations.

Strict constraints of the Higgs self-couplings for a 3-3-1 model with right handed neutrino were discussed in Ref. Sanchez-Vega:2018qje , where the Higgs potential is forced to satisfy the vacuum stability condition. Accordingly, interesting results can be applied to the 3-3-1 models with arbitrary $\beta$, namely

[TABLE]

with $i,j=1,2,3$ and $i<j$. Note that the constraints on the Higgs self-couplings $\lambda_{1,2,12}$ correspond to the particular cases of the 2HDM Tanabashi:2018oca ; Kanemura:2018yai ; Kanemura:1999xf . Because $\left(t_{12}+t^{-1}_{12}+c_{12}(v/v_{3})^{2}\right)fv_{3}\sim m_{A}^{2}$ being the squared mass of the $CP$-odd neutral Higgs boson, the requirement $m^{2}_{A}>0$ shows $f>0$ Okada:2016whh ; Dias:2009au . The other conditions guarantee that all squared Higgs masses must be positive and SM-like Higgs mass is identified with the experimental value. It can be seen in Eqs. (49)-(59) that all charged Higgs squared masses are always positive if all $\tilde{\lambda}_{ij}>0$, but their values seem very large. More interesting cases correspond to the existence of light charged Higgs bosons, which may contribute significant contributions to loop-induced decays of the SM-like Higgs bosons. Based on eq. (61), a first estimation suggests that $f$ has the same order with $SU(3)_{L}$ scale $v_{3}$, leading to the requirement that $\tilde{\lambda}_{12,13,23}<0$ for the existence of light charged Higgs bosons. Furthermore, the relation (73) results in a consequence that $\lambda_{13}$ will be small for the case of our interest with large $v_{3}\geq 3$ TeV and small $m_{h^{0}_{2}}$ around 1 TeV. In this case $f$ is also small, as we realize in the numerical investigation as well as it has been shown recently Palcu:2019nld . Taking this into account to the charged Higgs masses in Eqs.(54) and (59) we derive that the absolute values of negative values of $\tilde{\lambda}_{13,23}$ seems very small. In contrast, the appearance of a light charged Higgs $H^{\pm}$ allows negative $\tilde{\lambda}_{12}$ and rather large $|\tilde{\lambda}_{12}|$ that satisfy the inequality $\tilde{f}_{12}>0$ given in (101). We will consider the two separate cases: $\tilde{\lambda}_{ij}\geq 0$ with all $i>j$, $i,j=1,2,3$; and $\tilde{\lambda}_{12}<0$. The values of $\lambda_{13,23}$ are always chosen to get large absolute values of $F^{331}_{21,s}$, and/or $F^{331}_{21,sv}\equiv F^{331}_{21,svv}+F^{331}_{21,vss}$.

The above discussion allows us to choose the default values of unknown independent parameters as follows: $\beta=1/\sqrt{3}$, $s_{\delta}=0.01$, $\lambda_{1}=1$, $t_{12}=0.8$, $\tilde{\lambda}_{12}=\tilde{\lambda}_{13}=\tilde{\lambda}_{23}=0.1$, $m_{h^{0}_{2}}=1.2$ TeV, $m_{h^{0}_{3}}=1$ TeV, $v_{3}=14$ TeV, $m_{E_{a}}=m_{J_{a}}=1.5$ TeV. We choose the perturbative limit of Higgs self-couplings is $10$, which is a bit more strict than $4\pi$ 444 We thank the referee for reminding us this point.. In addition, depending on the particular discussions, changing any numerical values will be noted.

IV.1.1 Case 1: $\tilde{\lambda}_{12}\geq 0$

First, we focus on the 2HDM parameters. Fig. 2 and 3 illustrate numerically Higgs self-couplings and $f_{ij}$ as functions of $m_{h^{0}_{2}}$, and other independent parameters are fixed as $t_{12}=0.8$ and changing $s_{\delta}=\pm 10^{-2},\pm 5\times 10^{-2}$, which are significantly large.

For $s_{\delta}>0$, the $t_{12}$ is chosen large enough to satisfy $f_{12}>0$ and $m_{h^{0}_{2}}>1$ TeV. The parameters $\tilde{\lambda}_{12}\geq 0$ and negative $\tilde{\lambda}_{13,23}\rightarrow 0$ do not affect the quantities investigated in this figure. We conclude that the vacuum stability requirement $f_{12}>0$ gives strong upper bound on $m_{h^{0}_{2}}$, where larger $s_{\delta}$ gives smaller allowed $m_{h^{0}_{2}}$.

Fig.  3 illustrates allowed regions for $s_{\delta}<0$, where we choose $t_{12}=0.1$, enough small to allow $\lambda_{2}>0$ and $m_{h^{0}_{2}}>1$ TeV.

Again we derive that larger $|s_{\delta}|$ gives smaller upper bound of $m_{h^{0}_{2}}$.

In general, our scan shows that allowed $t_{12}$ and $s_{\theta}$ are affected the most strongly by $m_{h^{0}_{2}}$. As illustration, the Fig. 4 presents allowed regions of $t_{12}$ and $s_{\theta}$ with two fixed $m_{h^{0}_{2}}=1$ TeV and $2.5$ TeV.

It can be seen that larger $m_{h^{0}_{2}}$ results in smaller allowed $|s_{\theta}|$. The dashed black curves presenting constant values of $f_{12}$ will be helpful for the discussion on the case of $\tilde{\lambda}_{12}<0$. This is because the constraint from $\tilde{f}_{12}>0$ will be more strict than that from $f_{12}>0$ when $\tilde{\lambda}_{12}<0$, namely it will be equivalent to $f_{12}>|\tilde{\lambda}_{12}|$. Hence $f_{12}$ plays role as the upper bound of $|\tilde{\lambda}_{12}|$.

The allowed regions also depend on $\lambda_{1}$, see contour plots in Fig. 5 corresponding to $\lambda_{1}=0.5,5$.

It can be seen that $\lambda_{1}$ should be large enough to allow large $|s_{\theta}|$, see illustrations in Fig. 12 for $\lambda_{1}=0.1,10$ in appendix C.

In the case of large $|s_{\theta}|=0.02$, the allowed values $\lambda_{1}$ and $t_{12}$ are shown in Fig. 6.

It can be seen that only negative $s_{\theta}$ allows large $f_{12}$. The case of larger $|s_{\theta}|=0.05$ is shown in Fig. 13 of the appendix C. We can choose $m_{h^{0}_{2}}=1.2$ TeV so that $|s_{\theta}|=0.05$ is still allowed. Both large $|s_{\delta}|$ and $m_{h^{0}_{2}}$ give narrow allowed regions of $t_{12}$ and $\lambda_{1}$, and small $f_{12}$. For small $|s_{\delta}|<10^{-2}$, the allowed values of $m_{h^{0}_{2}}$ and $t_{12}$ will relax. But it will not result in much deviation from the SM prediction.

The left panel of Fig. 7 illustrates the contour plots with fixed $\beta=-1/\sqrt{3}$ for allowed values of $\delta\mu_{Z\gamma}$ corresponding to the noncolored✿ regions that satisfy the constraints of parameters and the recent experimental bound on $\delta\mu_{\gamma\gamma}$.

The right panel of Fig. 7 shows the contour plots of $R_{Z\gamma/\gamma\gamma}$, where the noncolored region satisfies $R_{Z\gamma/\gamma\gamma}\geq 2$. In this region, we can see that $|s_{\delta}|\sim\mathcal{O}(10^{-3})$ and negative. In addition, $\delta\mu_{\gamma\gamma}<0.04$. Hence, the current constraints $\mu_{\gamma\gamma}=0.99\pm 0.14$ predicts $|\delta\mu_{Z\gamma}|<0.15$ which is still smaller than the future sensitivity $\delta\mu_{Z\gamma}=\pm 0.23$ mentioned in Ref. Cepeda:2019klc . In addition, most of the allowed regions satisfy $0.8\leq\;R_{Z\gamma/\gamma\gamma}\leq\;2$, hence the approximation Br$(h\rightarrow\gamma\gamma)\simeq\mathrm{Br}(h\rightarrow\;Z\gamma)$ is accepted for simplicity in previous works.

For large $v_{3}=14$ TeV and recent uncertainty of the $\delta\mu_{\gamma\gamma}$, our investigation shows generally that the above discussions on the allowed regions as well as $R_{Z\gamma/\gamma\gamma}$ illustrated in Fig. 7 depend weakly on $\beta$. The results are also unchanged for lower bound of $v_{3}=8$ TeV which is allowed for $\beta=\pm 2/\sqrt{3}$. This property can be explained by the fact that, large $v_{3}\simeq 10$ TeV results in heavy charged gauge bosons $m_{Y},m_{V}$ having masses around 4 TeV, and the charged Higgs masses being not less than 1 TeV. As a by-product, one loop contributions from $SU(3)_{L}$ particles to $F^{331}_{21}$ and $F^{331}_{\gamma\gamma}$ are at least four orders smaller than the corresponding SM amplitudes $F^{\mathrm{SM}}_{21,\gamma\gamma}$, illustrations are given in table 8.

Here we use the SM amplitudes predicted by the SM, namely $\mathrm{Re}[F^{\mathrm{SM}}_{21}]=-5.6\times 10^{-5}\,[\mathrm{GeV}^{-1}]$ and $\mathrm{Re}[F^{\mathrm{SM}}_{\gamma\gamma}]=-3.09\times 10^{-5}\,[\mathrm{GeV}^{-1}]$, and ignore the tiny imaginary parts. We can see that both $\delta\mu_{Z\gamma}$ and $\delta\mu_{\gamma\gamma}$ depend strongly on $s_{\delta}$ and $t_{12}$. In contrast, the one-loop contributions from new particles are suppressed, as shown in the last line in table 8: suppressed $s_{\delta}$ results in $|\delta\mu_{Z\gamma}|\simeq 4|\delta\mu_{\gamma\gamma}|=0.8\%\ll 4\%$, which is even much smaller than the expected sensitivity of $\delta\mu_{\gamma\gamma}=4\%$. Anyway, it can be noted that $F^{331}_{21,sv}$ may significantly larger than $F^{331}_{21,v}$, hence both of them should be included simultaneously into the decay amplitude $h\rightarrow\;Z\gamma$ in general. Suppressed contributions of new particles to $\delta\mu_{Z\gamma,}$ are shown explicitly in the left panel of Fig. 7, where three constant curves $s_{\delta}=\delta\mu_{Z\gamma}=\delta\mu_{\gamma\gamma}=0$ are very close together.

For large and positive $\tilde{\lambda}_{12}$ and small $m_{h^{0}_{2}}$, one loop contributions from $H^{\pm}$ to $F^{331}_{21}$ and $F^{331}_{\gamma\gamma}$ are dominant but still not large enough to give significant deviations to $\delta\mu_{Z\gamma}$, see an illustration with suppressed $s_{\delta}=10^{-3}$ in the first line of table 9.

Here we always force $|\delta\mu_{\gamma\gamma}|\leq 4\%$ being the future sensitive of $\mu_{\gamma\gamma}$. On the other hand, large deviations can result from large $|s_{\delta}|$. In this case, all $\delta\mu_{Z\gamma,\gamma\gamma}$ and $s_{\delta}$ have the same signs.

Regarding $\beta=\sqrt{3}$ corresponding to the model discussed in Ref. Coriano:2018coq , where $v_{3}=3$ TeV is still accepted, the allowed regions change significantly, as illustrated in Fig. 8.

In particularly, the model gives more strict positive $s_{\delta}<0.03$. One-loop contributions from $SU(3)_{L}$ particles can give deviations up to few percent for both $\delta\mu_{Z\gamma}$, $\delta\mu_{\gamma\gamma}$, as shown in Fig. 8 that the two contours $\delta_{Z\gamma}=\delta\mu_{\gamma\gamma}=0$ distinguish with the line $s_{\delta}=0$. Interesting numerical values are illustrated in table 10.

We emphasize two important properties. First, one loop contributions from gauge $SU(3)_{L}$ bosons are dominant, which can give $\delta\mu_{\gamma\gamma}$ to reach the future sensitivity. Values of $F^{331}_{21,v}$ and $F^{331}_{21,sv}$ can have the same order of $10^{-3}$ compared with the SM part, but these contributions are not large enough to result in large deviation of $|\delta\mu_{Z\gamma}|>23\%$.

To finish the case of $\lambda_{ij}>0$ we mentioned above, we see that in this case all of the charged Higgs boson masses are order of $\mathrm{O}(1)$TeV and they have small couplings with $h$. For large $\lambda_{1}$, $\lambda_{12}$ and small $m_{h^{0}_{2}}=800$ GeV, there may give small $\delta\mu_{\gamma\gamma}$ but large $\delta\mu_{Z\gamma}$, see examples in table 11.

We stress here an interesting point that with the existence of new Higgs and gauge bosons, their contributions $F^{331}_{\gamma\gamma,s}$ and $F^{331}_{\gamma\gamma,v}$ to the decay amplitude $h\rightarrow\gamma\gamma$ may be destructive and the same order, hence keep the respective signal strength satisfying the small experimental constraint. Simultaneously, all of the contributions to the decay amplitude $h\rightarrow\,Z\gamma$ are constructive so that the deviation can be large. For the model with $\beta=\sqrt{3}$ and $v_{3}=3$ TeV, we can find this deviation can reach around $-10$, but this values is still far from the expected sensitive $\delta\mu_{Z\gamma}=\pm 23\%$ in the HL-LHC project. For the models with $v_{3}\geq 8$ TeV, heavy gauge contributions are suppressed, hence large contribution from charged Higgs bosons is dominant. Then, the constraint from $\delta\mu_{\gamma\gamma}$ will give more strict constraint on $\delta\mu_{Z\gamma}$ than that obtained from the experiments.

IV.1.2 Case 2: $\tilde{\lambda}_{12}<0$.

As we can see in eq. (49), negative $\tilde{\lambda}_{12}$ may result in small charged Higgs mass $m_{H^{\pm}}$. In addition, large $|\tilde{\lambda}_{12}|$ may give large coupling of this Higgs boson with the SM-like one, leading to large $|F^{331}_{21,s}|$ and $|F^{331}_{\gamma\gamma,s}|$. We will focus on this interesting case.

One of the conditions given in (101), namely $\tilde{f}_{12}>0$, will automatically satisfy if $f_{12}>0$ and $\tilde{\lambda}_{12}\geq 0$. In the case of $\tilde{\lambda}_{12}<0$, the inequality $\tilde{f}_{12}>0$ is equivalent to the more strict condition $f_{12}>|\tilde{\lambda}_{12}|>0$ or $-f_{12}<\tilde{\lambda}_{12}<0$. This helps us determine the allowed regions with large $|\tilde{\lambda}_{12}|$, which give large one-loop contributions of charged Higgs boson $H^{\pm}$ to the two decay amplitudes $h\rightarrow\,Z\gamma,\gamma\gamma$. Based on the fact that allowed regions with large positive $f_{12}$ will allow large $\tilde{\lambda}_{12}$, two Figs. 4 and 5 show that large $\tilde{\lambda}_{12}$ corresponds to regions having negative $s_{\delta}$ and small $t_{12}$. Small $s_{\theta}$ allows small $|\tilde{\lambda}_{12}|$. The Fig. 6 shows that values of $\lambda_{1}$ seems not affect allowed $\tilde{\lambda}_{12}$ in the regions of negative $s_{\delta}$. For large $v_{3}$, large $|\tilde{\lambda}_{12}|$ in this case does not affect significantly on both $\delta\mu_{Z\gamma,\gamma\gamma}$ see illustration in table 12.

Hence, the case of negative $\tilde{\lambda}_{12}$ may results in light charged Higgs boson $H^{\pm}$, but it does not support large one-loop contributions from charged Higgs mediation to decay amplitudes $h\rightarrow\,Z\gamma,\gamma\gamma$.

Regarding the model with $\beta=\sqrt{3}$ discussed in Ref. Coriano:2018coq , the main difference is the small $v_{3}=3$ TeV, leading to a significant effect of heavy gauge bosons to the one-loop contributions $|F^{331}_{21,v}/F^{\mathrm{SM}}_{21}|,\,|F^{331}_{21,sv}/F^{\mathrm{SM}}_{21}|,\,|F^{331}_{21,s}/F^{\mathrm{SM}}_{21}|,\,|F^{331}_{\gamma\gamma,v}/F^{\mathrm{SM}}_{\gamma\gamma}|,\,|F^{331}_{\gamma\gamma,s}/F^{\mathrm{SM}}_{\gamma\gamma}|\sim\mathcal{O}(10^{-2})$. But with $\tilde{\lambda}_{12}<0$, constructive contributions appear in the decay amplitude $h\rightarrow\gamma\gamma$, while destructive contributions appear in the decay amplitude $h\rightarrow\,Z\gamma$. Hence, the constraint from experimental data of the decay $h\rightarrow\gamma\gamma$ predicts smaller deviation of the $\mu_{Z\gamma}$ than that corresponding to $\tilde{\lambda}_{12}>0$.

To finish, from above discussion we emphasize that in other gauge extensions of the SM such as the $SU(2)_{1}\otimes SU(2)_{2}\otimes U(1)_{Y}$ models, which still allow low values of new gauge and charged Higgs bosons masses Boucenna:2016qad ; Hue:2016nya ; He:2017bft ; Yue:2019kky ; Abdullah:2018ets , the contributions like $F_{12,sv}$ may be as large as usual ones, hence it should be included in the decay amplitude $h\rightarrow\,Z\gamma$. In addition, these models may predict large $\delta\mu_{Z\gamma}$, which also satisfies $|\delta\mu_{\gamma\gamma}|\leq 0.04$. This interesting topic deserves to be paid attention more detailed.

IV.2 $h^{0}_{3}$ decays as a signal of the $331\beta$ model

Different contributions to loop-induced decays $h^{0}_{3}\rightarrow\gamma\gamma,Z\gamma$ with small $s_{\theta}=10^{-3}$, $m_{h^{0}_{3}}=700$ GeV, $t_{12}=0.8$ are illustrated in Fig. 9, where the ratios $|F_{21,x}(h^{0}_{3}\rightarrow\,Z\gamma)|/|F_{21}(h^{0}_{3}\rightarrow\,Z\gamma)|$ and $|F_{\gamma\gamma,x}(h^{0}_{3}\rightarrow\,Z\gamma)|/|F_{\gamma\gamma}(h^{0}_{3}\rightarrow\,Z\gamma)|$ are presented, $x=f,s,v,sv$.

In addition, our scan shows that the curves in the Fig. 9 do not sensitive with the changes of $s_{\delta}$. We can conclude that contributions from heavy exotic fermions are alway dominant for large $\beta$. While $F_{21,sv}$ is suppressed. For the decay $h^{0}_{3}\rightarrow\gamma\gamma$, the destructive correlation between $F_{\gamma\gamma,v}$ and $F_{\gamma\gamma,f}$ happens with small $|\beta|$. This results in two peaks in the figure, where $|F_{\gamma\gamma}|\ll|F_{\gamma\gamma,f}|,|F_{\gamma\gamma,v}|$.

Individual branching ratios of $h^{0}_{3}$ are shown in Fig. 10.

The most interesting property is that, the Br$(h^{0}_{3}\rightarrow\gamma\gamma)$ may have large values and it is very sensitive with the change of $\beta$. Hence this decay is a promising channel to fix the $\beta$ value once $h^{0}_{3}$ exists. On the other hand, Br$(h^{0}_{3}\rightarrow\,hh)$ is sensitive with $s_{\delta}$: it increases significantly with large $s_{\delta}$, but the values is always small Br$(h^{0}_{3}\rightarrow\,hh)<1\%$.

For $m_{F}>m_{h^{0}_{3}}$ the total decay width of the $h^{0}_{3}$ gets dominant contribution from two gluons decay channel, hence it is sensitive with only $m_{h^{0}_{3}}$ and $v_{3}$, as given in Eq. (94). It is a bit sensitive with $\beta$, see illustrations in Fig. 11.

V Conclusions

The signals of new physics predicted by the 3-3-1 models from the loop-induced neutral Higgs decays $h,h^{0}_{3}\rightarrow\gamma\gamma,Z\gamma$ have been discussed. For the general case with arbitrary $\beta$, we have derived that these decays of the SM-like Higgs boson do not depend on the $\beta$, i.e., they cannot be used to distinguish different models corresponding to particular $\beta$ values. This is because of the very large $v_{3}$ with values around $10$ TeV, leading to the suppressed one-loop contributions from heavy gauge and charged Higgs bosons, except the $H^{\pm}$, which are also predicted by the 2HDM and are irrelevant with $\beta$. Hence, the large deviations $\delta\mu_{Z\gamma,\gamma\gamma}$ originate from the one-loop contribution of the $H^{\pm}$ and large $|s_{\delta}|$. In the region resulting in large $\delta\mu_{Z\gamma}$, the recent constraint on the $\mu_{\gamma\gamma}$ always gives more strict upper bound on $\mu_{Z\gamma}$ than that obtained from recent experiments. In particular, our numerical investigation predicts $|\delta\mu_{Z\gamma}|\leq|\delta\mu_{\gamma\gamma}|<0.23$, which is the sensitivity of $\mu_{Z\gamma}$ given in HC-HL project.

On the other hand, in a model with $\beta=\sqrt{3}$, where $v_{3}\simeq 3$ TeV is still valid Coriano:2018coq , $\delta\mu_{Z\gamma}$ may be large in the allowed region $\mu_{\gamma\gamma}=0.99\pm 0.14$. For the near future HC-HL project, where the experimental sensitivity for the decay $h\rightarrow\gamma\gamma$ may reach $|\delta\mu_{\gamma\gamma}|=0.04$, this model still allows $|\delta\mu_{Z\gamma}|$ to be close to 0.1. But it cannot reach the near future sensitivity $|\delta\mu_{Z\gamma}|=0.23$.

Theoretically, we have found two very interesting properties. First, $F^{331}_{21,sv}$ may have order of $F^{331}_{21,v}$ in allowed regions of the parameter space. This happens also in the 3-3-1 model with $\beta=\sqrt{3}$, where loop contributions from gauge and Higgs bosons may be large and have the same order. Hence, $F^{331}_{21,sv}$ should not be ignored in previous treatments for simplicity  Yue:2013qba ; Cao:2016uur . Second, in the model with $\beta=\sqrt{3}$, one-loop contributions from gauge bosons can reach the order of charged Higgs contributions, leading to that there appear regions where different contributions to the amplitude $h\rightarrow\gamma\gamma$ are destructive, while they are constructive in contributing to the decay amplitude $h\rightarrow\,Z\gamma$. This suggests that there may exist recent gauge extensions of the SM that allow large $|\delta\mu_{Z\gamma}|$ while still satisfy the future experimental data including $|\delta\mu_{\gamma\gamma}|\leq 0.04$.

Finally, the $h^{0}_{3}$ being the $CP$-enven neutral Higgs boson predicted by the $SU(3)_{L}$ symmetry, not appear in the effective 2HDM. This Higgs boson couples to only SM-like Higgs through the Higgs self-couplings, while decoulpes to all other SM-like particles . If $h^{0}_{3}$ is the lightest among new particles, loop-induced decays $h^{0}_{3}\rightarrow\gamma\gamma,Z\gamma,gg$ are still allowed. Our investigation shows that the Br$(h\rightarrow\gamma\gamma)$ is very sensitive with the parameter $\beta$, hence it is a promising channel to distinguish different 3-3-1 models. Because of the strong Yukawa couplings with new heavy fermions, $h^{0}_{3}$ can be produced through the gluon fusion in the future project HL-LHC.

Acknowledgments

This research is funded by the Ministry of Education and Training of Vietnam under grant number: B.2018-SP2-12.

Appendix A Heavy neutral Higgs couplings

From the Higgs potential and the aligned limit (61), the triple Higgs couplings containing one heavy neutral Higgs boson $h^{0}_{3}$ are listed in table 13. We only mention the couplings relating with discussion on the decays $h^{0}_{3}\rightarrow\gamma\gamma,Z\gamma$.

The nonzero couplings of heavy neutral Higgs bosons with gauge bosons are listed in table 14. They are derived from the Lagrangian given in (III.1), exactly the same way used to calculate the similar couplings of the SM-like Higgs boson. Hence, the notations for the couplings of these heavy Higgs bosons are the replacements $h\rightarrow h^{0}_{2},h^{0}_{3}$ in those given in Eq. (III.1).

The couplings of $Z$ to two exotic fermions are given in table 16.

Appendix B Form factors to one-loop amplitudes of the neutral Higgs decays $h,h^{0}_{3}\rightarrow Z\gamma,\gamma\gamma$

In the $331\beta$ model, the explicit analytic formulas of one-loop contributions to the anplitudes of the decay $h\rightarrow\gamma\gamma,Z\gamma$ will be presented in terms of the Passarino-Veltmann (PV) functions Passarino:1978jh , namely the one-loop three point PV functions denoted as $C_{i}$ and $C_{ij}$ with $i,j=0,1,2$. The particular forms for one-loop contributions to the decay amplitudes $h\rightarrow Z\gamma,\gamma\gamma$ were given in Ref. Hue:2017cph , which are consistent with the previous formulas Degrande:2017naf . We have used the LoopTools Hahn:1998yk to evaluate numerical results.

For the loop-induced decays of the heavy neutral Higgs bosons $h^{0}_{3}$, the calculation is the same way as those for the SM-like Higgs boson $h$. Correspondingly, the mass and couplings of $h$ are replaced with those relating with $h^{0}_{3}$. The $h^{0}_{2}$ properties were discussed in ref. Okada:2016whh , we do not repeat again.

The contributions from the SM fermions corresponding to the diagram 1 in Fig. 1 are

[TABLE]

where $C_{0,i,ij}\equiv C_{0,i,ij}(m_{Z}^{2},0,m_{h}^{2};m_{f}^{2},m_{f}^{2},m_{f}^{2})$; $Q_{f}$ $N_{c}$ and $m_{f}$ are respectively the electric charge, color factor and mass of the SM fermions. The factors $Y_{h\bar{f}fL}$ and $g^{f}_{L,R}$ are listed in tables 1 and 5, respectively.

The contributions from the charged Higgs bosons $s=H^{\pm},H^{\pm A},H^{\pm B}$ corresponding to the diagram 2 in Fig. 1 are

[TABLE]

where $s=H^{\pm},H^{\pm A},H^{\pm B}$, $C_{0,i,ij}\equiv C_{0,i,ij}(m_{Z}^{2},0,m_{h}^{2};m_{s}^{2},m_{s}^{2},m_{s}^{2})$, and the couplings $\lambda_{hss},g_{Zss}$ are listed in table 1 and 4.

The contributions from the diagrams containing both charged Higgs and gauge bosons $\{v,s\}=\{W^{\pm},H^{\pm}\},\{Y^{\pm A},H^{\pm A}\},\{V^{\pm B},H^{\pm B}\}$ corresponding to the two diagrams 3 and 4 in Fig. 1 are

[TABLE]

where $C_{0,i,ij}\equiv C_{0,i,ij}(m_{Z}^{2},0,m_{h}^{2};m_{V}^{2},m_{s}^{2},m_{s}^{2})$ or $C_{0,i,ij}(m_{Z}^{2},0,m_{h}^{2};m_{s}^{2},m_{V}^{2},m_{V}^{2})$ corresponding to Eqs. (104) or (105). The vertex factors are listed in table 3 and 4.

The contributions from the charged gauge bosons $v=W^{\pm},Y^{\pm A},V^{\pm B}$ corresponding to the diagram 5 in Fig. 1 are

[TABLE]

where $v=W^{\pm},Y^{\pm A},V^{\pm B}$, $C_{0,i,ij}\equiv C_{0,i,ij}(m_{Z}^{2},0,m_{h}^{2};m_{v}^{2},m_{v}^{2},m_{v}^{2})$. The vertex factors are listed in table 3 and 6.

For the decay $h\rightarrow\gamma\gamma$, analytic formulas of $F^{331}_{\gamma}$ can be derived from the $F^{331}_{21}$ by taking replacements $g_{Zvv},g_{Zss},\frac{gc_{\theta}}{c_{W}}g^{f}_{L,R}\rightarrow eQ_{v},eQ_{s},eQ_{f}$ and the respective PV functions, name ly:

[TABLE]

where $C_{0,i,ij}\equiv C_{0,i,ij}(0,0,m_{h}^{2};m_{x}^{2},m_{x}^{2},m_{x}^{2})$ with $x=f,s,v$ corresponding to the contribution from fermions, charged Higgs and gauge bosons.

Regarding to $h^{0}_{3}$, we emphasize again that the only nonzero coupling with SM particle is the triple couplings with two SM-like Higgs bosons. Hence the fermion contributions to the decay amplitudes $h^{0}_{3}\rightarrow\gamma\gamma,Z\gamma,gg$ are only exotic fermions $F=E_{a},J_{a}$. These contributions are denoted as $F^{331,h^{0}_{3}}_{\gamma\gamma,F},F^{331,h^{0}_{3}}_{21,F},F^{331,h^{0}_{3}}_{gg,F}$. They are derived base on Eq. (81) with the following replacement,

[TABLE]

The other contributions to the mentioned $h^{0}_{3}$ decays are calculated by simple replacements the mass and couplings of the SM-like Higgs bosons with those of the $h^{0}_{3}$. We note that the $W$ bosons are not included in these amplitudes.

Appendix C More numerical illustrations discussed in section IV

Contour plots with other numerical values of $\lambda_{1}$ (Fig. 12) and $|s_{\theta}=0.05|$ (figure 13).