LHC Constraints on a $(B-L)_3$ Gauge Boson

TL;DR

This paper proposes a new LHC search strategy for a third-generation coupled gauge boson in the 10 to 2000 GeV mass range, improving existing constraints and exploring previously unconstrained parameter space.

Contribution

It develops a dedicated search method for a third-generation $B-L$ gauge boson at the LHC, focusing on the $b\bar{b}\tau^+\tau^-$ channel, and provides projected sensitivity improvements.

Findings

Detects $X$ gauge boson with couplings $g_X \sim 0.005-0.01$ for $m_X<50$ GeV

Achieves sensitivity to $g_X \sim 0.1$ for heavier $X$ with 100 fb$^{-1}$

Improves constraints by a factor of 2-10 over previous bounds

Abstract

In this paper, we explore the constraints that the LHC can place on a massive gauge boson $X$ that predominantly couples to the third generation of fermions. Such a gauge boson arises in scenarios where the $B-L$ of the third generation is gauged. We focus on the mass range $10 \leq m_X \lesssim 2\,m_W$, where current constraints are lacking, and develop a dedicated search strategy. For this mass range, we show that $b \bar b \tau^+ \tau^-$, where at least one of the $\tau$s decay leptonically is the optimal channel to look for the $X$ at the LHC. The QCD production of $b$ quarks, combined with the cleanliness of the leptons coming from the decay of the $\tau$ allow us to detect $X$ gauge boson with couplings of $g_X \sim(0.005-0.01)$, for $m_X < 50 \ \text{GeV}$, and a coupling of $O(0.1)$ for heavier $X$ gauge boson with $100 \ \text{fb}^{-1}$ of integrated luminosity. This is about a factor of 2-10 improvement over previous constraints coming from the decay of $\Upsilon \to \tau^+ \tau^-$. Extrapolating to the full HL-LHC luminosity of $3000 \ \text{fb}^{-1}$, the bounds on $g_X$ can be enhanced by another factor of $\sqrt{2}$ for $m_X < 50 \ \text{GeV}$.

Figures

Tables

Table 1. Table 1 : Scalar and fermion fields charged under the U ​ ( 1 ) B − L ( 3 ) 𝑈 superscript subscript 1 𝐵 𝐿 3 U(1)_{B-L}^{(3)} gauge symmetry.

		$SU{\left(3\right)}_c\times SU{\left(2\right)}_{\text{W}}\times U{\left(1\right)}_Y$		$U{\left(1\right)}_{B-L}^{\left(3\right)}$
$\varphi$		$(\mathrm{𝟏},\mathrm{𝟐},1/2)$		$1/3$
$s$		$(\mathrm{𝟏},\mathrm{𝟏},0)$		$1/3$
$Q_{3L}$		$(\mathrm{𝟑},\mathrm{𝟐},-1/6)$		$1/3$
$t_R$		$(\mathrm{𝟑},\mathrm{𝟏},2/3)$		$1/3$
$b_R$		$(\mathrm{𝟑},\mathrm{𝟏},-1/3)$		$1/3$
$L_{3L}$		$(\mathrm{𝟏},\mathrm{𝟐},-1/2)$		$-1$
${\tau }_R$		$(\mathrm{𝟏},\mathrm{𝟏},-1)$		$-1$
${\nu }_{\tau R}$		$(\mathrm{𝟏},\mathrm{𝟏},0)$		$-1$

Table 2. Table 2 : The effect of each cut on the signal benchmark with m X = 30 ​ GeV subscript 𝑚 𝑋 30 GeV m_{X}=30\leavevmode\nobreak\ \mathrm{GeV} and each of the backgrounds.

cuts	$X_{\mu }$	SM backgrounds (fb)
	$(30\text{GeV},0.02)$	$\begin{array}{c}b\overline{b}{\ell }^{+}{\ell }^{-}\\ c\overline{c}{\ell }^{+}{\ell }^{-}\\ j_l{j}_l{\ell }^{+}{\ell }^{-}\end{array}$	$\begin{array}{c}b\overline{b}VV\\ c\overline{c}VV\\ j_l{j}_{l}VV\end{array}$	$\begin{array}{c}b\overline{b}{\tau }^{+}{\tau }^{-}\\ c\overline{c}{\tau }^{+}{\tau }^{-}\\ j_l{j}_l{\tau }^{+}{\tau }^{-}\end{array}$	${\text{significance}|}_{ℒ=100{\mathrm{fb}}^{-1}}$
basic selection	8.13	$\begin{array}{c}254.3\\ 20.3\\ 8\end{array}$	$\begin{array}{c}1362\\ 2\\ 1.8\end{array}$	$\begin{array}{c}27.8\\ 2.4\\ 2\end{array}$	0.005
	7.9	$\begin{array}{c}2.6\\ 0.85\\ 0.5\end{array}$	$\begin{array}{c}12.44\\ 0.2\\ 0.18\end{array}$	$\begin{array}{c}0.16\\ 0.06\\ 0.02\end{array}$	5.77
	6.4	$\begin{array}{c}0.3\\ 0.09\\ 0.2\end{array}$	$\begin{array}{c}0.44\\ 0.0\\ 0.0\end{array}$	$\begin{array}{c}0.13\\ 0.04\\ 0.02\end{array}$	40.0
	5.48	$\begin{array}{c}0.09\\ 0.02\\ 0.1\end{array}$	$\begin{array}{c}0.3\\ 0.0\\ 0.0\end{array}$	$\begin{array}{c}0.07\\ 0.02\\ 0.02\end{array}$	49.0
efficiencies	$67.4\%$	$\begin{array}{c}0.03\%\\ 0.01\%\\ 0.1\%\end{array}$	$\begin{array}{c}0.02\%\\ 0.0\%\\ 0.0\%\end{array}$	$\begin{array}{c}0.3\%\\ 0.8\%\\ 1\%\end{array}$

Table 3. Table 3 : The cut-flow for m X = 10 ​ GeV subscript 𝑚 𝑋 10 GeV m_{X}=10\leavevmode\nobreak\ \mathrm{GeV} . The cross section of this benchmark is small after the basic selection. However, the m ℓ ​ ℓ subscript 𝑚 ℓ ℓ m_{\ell\ell} cut efficiently enhances the signal to background ratio.

cuts	$X_{\mu }$	SM backgrounds (fb)			${\text{significance}|}_{ℒ=100{\mathrm{fb}}^{-1}}$
	$(10\text{GeV},0.02)$	$b\overline{b}{\ell }^{+}{\ell }^{-}$	$b\overline{b}{V}_{\ell +{\tau }_{\ell }}{V}_{\ell +{\tau }_{\ell }}$	$b\overline{b}{\tau }^{+}{\tau }^{-}$
basic selection	0.61	282.6	1365.8	32.2	0.005
	0.5	0.5	1.55	0.03	2.3
	0.3	0.05	0.26	0.03	4.6
	0.3	0.03	0.21	0.03	5.3
efficiencies	$50\%$	$0.01\%$	$0.016\%$	$0.1\%$

Table 4. Table 4 : The effect of each cut on the signal with m X = 20 ​ GeV subscript 𝑚 𝑋 20 GeV m_{X}=20\leavevmode\nobreak\ \mathrm{GeV} and the backgrounds. The m ℓ ​ ℓ subscript 𝑚 ℓ ℓ m_{\ell\ell} cut efficiently enhances the signal to background ratio.

cuts	$X_{\mu }$	SM backgrounds (fb)			${\text{significance}|}_{ℒ=100{\mathrm{fb}}^{-1}}$
	$(20\text{GeV},0.02)$	$b\overline{b}{\ell }^{+}{\ell }^{-}$	$b\overline{b}{V}_{\ell +{\tau }_{\ell }}{V}_{\ell +{\tau }_{\ell }}$	$b\overline{b}{\tau }^{+}{\tau }^{-}$
basic selection	2.27	282.6	1365.8	32.2	0.02
	2.04	1.88	12.33	0.16	2.00
	1.69	0.23	0.44	0.13	12.9
	1.67	0.06	0.30	0.07	21.0
efficiencies	$73.2\%$	$0.02\%$	$0.02\%$	$0.27\%$

Table 5. Table 5 : The effect of each cut on the signal with m X = 30 ​ GeV subscript 𝑚 𝑋 30 GeV m_{X}=30\leavevmode\nobreak\ \mathrm{GeV} and the backgrounds. The m ℓ ​ ℓ subscript 𝑚 ℓ ℓ m_{\ell\ell} cut efficiently enhances the signal to background ratio.

cuts	$X_{\mu }$	SM backgrounds (fb)			${\text{significance}|}_{ℒ=100{\mathrm{fb}}^{-1}}$
	$(30\text{GeV},0.02)$	$b\overline{b}{\ell }^{+}{\ell }^{-}$	$b\overline{b}{V}_{\ell +{\tau }_{\ell }}{V}_{\ell +{\tau }_{\ell }}$	$b\overline{b}{\tau }^{+}{\tau }^{-}$
basic selection	8.13	282.6	1365.8	32.2	0.07
	7.9	3.95	12.82	0.24	5.77
	6.4	0.32	0.44	0.19	40.0
	5.48	0.21	0.31	0.11	49.0
efficiencies	$67.4\%$	$0.04\%$	$0.01\%$	$0.3\%$

Table 6. Table 6 : The effect of each cut on the signal with m X = 50 ​ GeV subscript 𝑚 𝑋 50 GeV m_{X}=50\leavevmode\nobreak\ \mathrm{GeV} and the backgrounds.

cuts	$X_{\mu }$	SM backgrounds (fb)			${\text{significance}|}_{ℒ=100{\mathrm{fb}}^{-1}}$
	$(50\text{GeV},0.02)$	$b\overline{b}{\ell }^{+}{\ell }^{-}$	$b\overline{b}{V}_{\ell +{\tau }_{\ell }}{V}_{\ell +{\tau }_{\ell }}$	$b\overline{b}{\tau }^{+}{\tau }^{-}$
basic selection	3.42	282.6	1365.8	32.2	0.03
	2.88	6.35	163	5.58	0.27
	2.08	0.74	7.07	3.38	1.37
	1.75	0.03	6.5	0.3	3.5
efficiencies	$51\%$	$0.01\%$	$0.48\%$	$1.17\%$

Table 7. Table 7 : The effect of each cut on the signal with m X = 80 ​ GeV subscript 𝑚 𝑋 80 GeV m_{X}=80\leavevmode\nobreak\ \mathrm{GeV} and the backgrounds. To further enhance the signal to background ratio, M T ​ 2 subscript 𝑀 𝑇 2 M_{T2} and m b ​ ℓ subscript 𝑚 𝑏 ℓ m_{b\ell} is used.

cuts	$X_{\mu }$	SM backgrounds (fb)			${\text{significance}|}_{ℒ=100{\mathrm{fb}}^{-1}}$
	$(80\text{GeV},0.02)$	$b\overline{b}{\ell }^{+}{\ell }^{-}$	$b\overline{b}{V}_{\ell +{\tau }_{\ell }}{V}_{\ell +{\tau }_{\ell }}$	$b\overline{b}{\tau }^{+}{\tau }^{-}$
basic selection	2.8	282.6	1365.8	32.2	0.02
	2.51	2.58	317.2	21.25	0.11
	2.24	1.6	41.3	17.3	0.30
	1.15	0.56	10.3	8.8	0.35
efficiencies	$41.1\%$	$0.2\%$	$0.8\%$	$27.3\%$

Equations

$$s_X\equiv \frac{2}{3}\frac{g_X}{\sqrt{g^2+g^{{}^{\prime }2}}}\frac{v_{\varphi }^2}{v^2},$$

$$m_X^2=\frac{1}{9}{g}_X^2\left(\frac{v_{\varphi }^2{v}_h^2}{v^2}+v_s^2\right)=\frac{1}{9}{g}_X^2\left(v^2\frac{{\tan }^2\beta }{(1+{\tan }^2\beta {)}^2}+v_s^2\right).$$

$$c_{\alpha }=g_X{q}_{\alpha }^X+s_X\sqrt{g^2+g^{{}^{\prime }2}}{q}_{\alpha }^Z=g_X\left[q_{\alpha }^X+\frac{2}{3}{q}_{\alpha }^Z(1+\tan {\beta }^2{)}^{-1}\right].$$

$$\sigma (pp\to b\bar{b}{\tau }^{+}{\tau }^{-}{)}_{{}_{\text{Signal}}}\sim c_b(g_X,\tan \beta {)}^2\times f(m_X,s)\times \text{Br}(X\to {\tau }^{+}{\tau }^{-}{)}_{\tan \beta },$$

$${\sigma }_{{}_{\text{Signal}}}^{\text{New}}\sim {\sigma }_{{}_{\text{Signal}}}^{\text{Old}}{\left(\frac{c_b(g_X^n,\tan {\beta }^n)}{c_b(g_X^o,\tan {\beta }^o)}\right)}^2\times \frac{\text{Br}(X\to {\tau }^{+}{\tau }^{-}{)}_{\tan {\beta }^n}}{\text{Br}(X\to {\tau }^{+}{\tau }^{-}{)}_{\tan {\beta }^o}},$$

$$1.)$$

$$2.)$$

$$3.)$$

$$\displaystyle 2.)pp\to b\bar{b}\,\tau^{+}_{\ell}\tau^{-}_{\ell}\Rightarrow\left\{\begin{array}[]{c}pp\to c\bar{c}\tau^{+}_{\ell}\tau^{-}_{\ell}\\ pp\to j_{l}\,j_{l}\tau^{+}_{\ell}\tau^{-}_{\ell}\end{array}\right.$$

$$S\equiv \text{Luminosity}\times$$

$$B\equiv \text{Luminosity}\times$$

$$+\sigma (pp\to b\bar{b}{\tau }_{{}_{\ell }}^{+}{\tau }_{{}_{\ell }}^{-}{)}_{SM}\times {\kappa }_{b\bar{b}{\tau }^{+}{\tau }^{-}}$$

$$+\sigma (pp\to b\bar{b}{\ell }^{+}{\ell }^{-})\times {\kappa }_{b\bar{b}{\ell }^{+}{\ell }^{-}}\}.$$

$$\text{Significance}=\frac{S}{\delta B}=\frac{S}{\sqrt{B+{\sum }_i{\lambda }_i^2{B}^2}},$$

$$M_{T2}={\text{min}}_{{\nu }_T^1+{\nu }_T^2=/E_T}(\text{max}(M_T({\ell }^{+},{\nu }_1),M_T({\ell }^{-},{\nu }_2))),$$

$${\chi }^2=\frac{(m_{{\ell }_i{\nu }_1}^2-m_{{}_W}^2{)}^2}{{\sigma }_{{}_W}^4}+\frac{(m_{{\ell }_j{\nu }_2}^2-m_{{}_W}^2{)}^2}{{\sigma }_{{}_W}^4}+\frac{(m_{b_k{\ell }_i{\nu }_1}^2-m_t^2{)}^2}{{\sigma }_t^4}+\frac{(m_{b_n{\ell }_j{\nu }_1}^2-m_t^2{)}^2}{{\sigma }_t^4},$$

$$1.)$$

$$2.)$$

$$3.)$$

$$4.)$$

$$3.)pp\to b\bar{b}\,\ W^{\pm}_{\ell+\tau_{\ell}}+\text{jets}\Rightarrow\left\{\begin{array}[]{c}pp\to c\bar{c}\ W^{\pm}_{\ell+\tau_{\ell}}+\text{jets}\\ pp\to j_{l}\,j_{l}W^{\pm}_{\ell+\tau_{\ell}}+\text{jets}\end{array}\right.,$$

$$S$$

$$B$$

$$=\frac{S}{\delta B}=\frac{S}{\sqrt{B+{\lambda }^2{B}^2}}.$$

$$\L^{q}_{yuk}=\overline{\mathbf{Q}}_{L}\left(\begin{array}[]{ccc}y_{11}^{u}\widetilde{H}&y_{12}^{u}\widetilde{H}&y_{13}^{u}\widetilde{\phi}\\ y_{21}^{u}\widetilde{H}&y_{22}^{u}\widetilde{H}&y_{23}^{u}\widetilde{\phi}\\ 0&0&y_{33}^{u}\widetilde{H}\\ \end{array}\right)\mathbf{u}_{R}+\overline{\mathbf{Q}}_{L}\left(\begin{array}[]{ccc}y_{11}^{d}H&y_{12}^{d}H&0\\ y_{21}^{d}H&y_{22}^{d}H&0\\ y_{31}^{d}\phi&y_{32}^{d}\phi&y_{33}^{d}H\\ \end{array}\right)\mathbf{d}_{R}+\text{h.c.}$$

$$R_{12}^{uL}.M_{u}.R_{12}^{uR\dagger}=\left(\begin{array}[]{ccc}m_{u}^{0}&0&c\,m_{t}^{0}\\ 0&m_{c}^{0}&d\,m_{t}^{0}\\ 0&0&m_{t}^{0}\end{array}\right)\quad\text{and}\quad R_{12}^{dL}.M_{d}.R_{12}^{dR\dagger}=\left(\begin{array}[]{ccc}m_{d}^{0}&0&0\\ 0&m_{s}^{0}&0\\ am_{b}^{0}&bm_{b}^{0}&m_{b}^{0}.\end{array}\right),$$

$${\mathcal{L}}_{{}_{XQQ}}^{\text{tree}}\simeq \frac{g_X}{3}{\bar{u}}_L\left(\begin{array}{ccc}{V}_{ub}^2& V_{ub}{V}_{cb}& V_{ub}\\ V_{ub}{V}_{cb}& V_{cb}^2& V_{cb}\\ V_{ub}& V_{cb}& 1\end{array}\right){\gamma }^{\mu }{u}_L{X}_{\mu }+\frac{g_X}{3}{\bar{d}}_L\left(\begin{array}{ccc}0& 0& 0\\ 0& 0& 0\\ 0& 0& 1\end{array}\right){\gamma }^{\mu }{d}_L{X}_{\mu }.$$

Full text

I Introduction

Even though numerous experimental measurements attest to the validity of the Standard Model (SM), some enigmatic observations such as dark matter and neutrino masses compel us to look for new physics (NP) beyond the SM. Many NP models propose augmenting the SM gauge groups by a new gauge symmetry, with $U(1)$ being a popular choice. On the other hand, it is well-known that the SM Lagrangian respects some global $U(1)$ symmetries that are not demanded beforehand. Some of these so called “accidental” symmetrie are anomaly free and can be gauged, either within the SM alone or with minimal extension Foot:1990mn ; He:1990pn ; He:1991qd ; Foot:1994vd . Given that the nature already approves of the SM, it is worthwhile to explore extending the SM using its own suggested symmetries.

Among the possible symmetries, new interactions that involve electrons or the first two generation of quarks are severely constrained in various collider searches Pati:1974yy ; Marshak:1979fm ; Wilczek:1979et ; Mohapatra:1980qe ; Nelson:2007yq ; Harnik:2012ni and low energy experiments delAmoSanchez:2010bt ; Babu:2009nn ; Babu:1999me ; Golowich:2009ii . The $U(1)_{L_{\mu}-L_{\tau}}$ symmetry – the difference between muon and tau number – has also received a lot of attention in recent years  He:1991qd ; Baek:2001kca ; Ma:2001md ; Salvioni:2009jp ; Heeck:2011wj ; Harigaya:2013twa ; Carone:2013uh ; Altmannshofer:2014cfa ; Farzan:2015doa ; Farzan:2015hkd ; Elahi:2017ppe ; Elahi:2015vzh ; Chun:2018ibr , and much of its parameter space is already being probed. That leaves us with new gauge bosons that interact predominantly with the third generation of fermions. One such possibility is $U(1)_{B-L}^{(3)}$, the difference between baryon number and lepton number of the third generation, which is anomaly free and guageable provided we augment the SM by a right handed neutrino.

The $U(1)_{B-L}^{(3)}$ extension of the SM was first proposed by 1705.01822 to explain the flavor alignment of the third generation of quarks – the empirical observations that the mixings of the third generation of quarks with the other two generations are very small. Distinguishing the third generation by assigning it new quantum numbers under an additional symmetry prohibits the mixing with the third generation of quarks, and thus justifies its flavor alignment. Of course, the symmetry needs to be broken at some scale to allow small, yet non-zero mixing between the generations Dev:2018pjn .

To achieve non-zero mixing between the generations at low scales, the $U(1)_{B-L}^{(3)}$ symmetry needs to be spontaneously broken by a scalar $\phi$ that is charged under $U(1)_{B-L}^{(3)}\times\mathcal{G}_{SM}$ and acquires a vacuum expectation value (vev). For certain charge assignments and coupling structure, it is possible to generate a realistic CKM matrix while relegating all tree-level flavor-changing-neutral currents (FCNC) to the up-quark sector 1705.01822 (see also Appendix B). The up-sector FCNC are suppressed by powers of CKM elements, however they – along with the down-sector FCNC they generate at loop level – are constrained by multiple low energy experiments. The constraints from experiments such as BaBar delAmoSanchez:2010bt , E949 Anisimovsky:2004hr ; Artamonov:2008qb , BESIII Ablikim:2014uzh , and CHARM-II Vilain:1994qy are severe, but peter out once $m_{X}\gtrsim 5\,\leavevmode\nobreak\ \mathrm{GeV}$. Furthermore, the direct coupling of $X$ with third generation of fermions can also contribute to the decay of $\Upsilon\to\tau\tau$, which constrains the available parameter space for $X$ near $m_{X}\sim m_{\Upsilon}\simeq 10\leavevmode\nobreak\ \mathrm{GeV}$ 1002.4358 , however the contribution of off-shell $X$ to $\Upsilon$ decay dies off rapidly as we move away from $m_{\Upsilon}$.

The $m_{X}$ window $\gtrsim 10\,\leavevmode\nobreak\ \mathrm{GeV}$ is only loosely constrained and is therefore the focus of this study. In practice, we impose an upper limit of $m_{X}<2m_{W}$, as the interaction of $X$ with $W$ gauge bosons is closely tied to the mixing angle between Higgs and the $U(1)_{B-L}^{(3)}$ breaking scalars $\phi$, and thus introduces multiple additional parameters; $X$ in this range could conceivably be constrained by LHC resonant diboson searches such as Ref. Aaboud:2017fgj ; Sirunyan:2017acf ; Sirunyan:2017nrt ; CMS:2017skt ; Biesuz:2017zip . For even larger $m_{X}>2m_{t}$, $X$ phenomenology is driven by decays to top pairs. In this sense, $X$ phenomenology can be mapped into $Z^{\prime}\to t\bar{t}$ searches, which have been studied extensively Hill:1991at ; Hill:1993hs ; Hill:1994hp ; Harris:2011ez ; Rosner:1996eb ; Lynch:2000md ; Carena:2004xs ; Choudhury:2007ux ; Khachatryan:2015sma ; CMS:2018ohu ; Aaboud:2018mjh ; Sirunyan:2017uhk ; Sirunyan:2017yar ; Cerrito:2016qig ; Arina:2016cqj ; Pedersen:2015knf ; Fox:2018ldq .

Having selected the $X$ mass window we are interested in, the next step is to determine the optimal LHC $X$ production mode and decay channel. As $X$ has suppressed couplings to first and second generation fermions, we either have to rely on the $b$ parton distribution function (PDF) (for $pp\to X$), or to produce the $X$ in association with third generation fermions, e.g. $pp\to\bar{f}fX$ where $f=t/b/\tau/\nu_{\tau}$. The PDF of b-quark is small, therefore we focus on associated production111While smaller, we do include processes initiated by $b$ PDFs in all our analyses.. The production of colored objects at the LHC is significantly larger than leptons, therefore we will concentrate on the scenario where $X$ is produced in association with a pair of $b$ quarks. Associated production of $X$ with top quarks is also an option, but suffers in rate due to the increased energy requirement as well as in reconstruction complexity, so we do not consider it here.

Turning to $X$ decay, if $X$ decays to a pair of $b$ quarks, we have a four b final state, which makes QCD backgrounds overwhelming and introduces a combinatorics problem. Among the leptonic decays of $X$, $\tau$s are more preferable because they give more handles for kinematic variables. Thereby, we settle on $pp\to b\bar{b}X\to b\bar{b}\tau^{+}\tau^{-}$.

One may think that further focusing the search on the $Z$ resonance contribution is a useful way to suppress backgrounds, as done in Ref. Elahi:2015vzh for the case of $U(1)_{L_{\mu}-L_{\tau}}$ gauge bosons in $pp\to 4\,\mu$. However, the poorer energy resolution for jets (as compared to muons in Ref. Elahi:2015vzh ) and the inevitable missing energy from neutrinos in tau decay hamper this technique and we find it is more beneficial to focus on QCD-produced $b\bar{b}$ pairs that emit an $X\to\tau^{+}\tau^{-}$.

The channel $b\bar{b}\tau^{+}\tau^{-}$ has already attracted some attention at the LHC in the search for the third generation leptoquarks  CMS:2018pab ; Khachatryan:2014ura and di-Higgs searches Sirunyan:2017tqo ; Sirunyan:2018two ; Cadamuro:2017wma . However, due to their particular optimized cuts, these analyses will have limited-to-no sensitivity to $X$ in our mass range of interest. More specifically,

• •

The search for the third generation leptoquarks  CMS:2018pab ; Khachatryan:2014ura is ineffective because they impose $m_{\tau b}>250\leavevmode\nobreak\ \mathrm{GeV}$, whereas we find that our signal prefers $m_{\tau b}<150\leavevmode\nobreak\ \mathrm{GeV}$ for $m_{X}\leq 2\,m_{W}$.

• •

the CMS di-Higgs search Sirunyan:2017tqo considers $b\bar{b}\tau^{+}\tau^{-}$ in the mass window $400\leavevmode\nobreak\ \mathrm{GeV}<m_{bb\tau\tau}<700\leavevmode\nobreak\ \mathrm{GeV}$. In our signal, however, the production of $b\bar{b}X$ is maximum at threshold, which means even for $m_{X}=2\,m_{W}$, we expect most of our events to lie in the $m_{bb\tau\tau}<350\leavevmode\nobreak\ \mathrm{GeV}$ region.

• •

the results of other CMS di-Higgs searches Sirunyan:2018two ; Cadamuro:2017wma are not easily recastable because they use boosted-decision-tree (BDT).

Given the lack of constraints from the current LHC searches or any other experiments, in the following sections we develop a LHC search strategy for $U(1)_{B-L}^{(3)}$ gauge bosons, $10\,\leavevmode\nobreak\ \mathrm{GeV}\leq m_{X}\leq 2\,m_{W}$, using the $b\bar{b}\tau^{+}\tau^{-}$ final state. We will assume throughout that $X$ is short-lived and therefore focus on prompt signals. Long-lived $X$, leading to displaced vertices at the LHC may be interesting to study, but likely require extending the setup in some way222In the current setup, the $X$ lifetime and production rate are governed by the same coupling, so one cannot make the particle long-lived without killing the production rate.. For the case of fully hadronic $b\bar{b}\tau^{+}\tau^{-}$ (prompt), the QCD backgrounds are overwhelming. Therefore, we will narrow our attention to semi-leptonic and fully leptonic decays of $\tau$s. Despite the large SM backgrounds (e.g., $t\bar{t}\to b\bar{b}W^{\pm}W^{\pm}\to b\bar{b}\tau^{+}\tau^{-}+\not{E}_{T}$), we show that the LHC-13 TeV, with the currently luminosity, can significantly improve the bounds on the $U(1)_{B-L}^{(3)}$ gauge coupling $g_{X}$.

The organization of the rest of the paper is as follows. In the upcoming section (Sec. II), we introduce the model, including the free parameters we will consider for the phenomenology of $X$ gauge boson at the LHC. Next, in Sec. III, we explore the LHC power in improving the bounds using simple kinematic variables – both for $m_{X}<m_{Z}$ (Sec. III.1) and for slightly heavier $m_{X}\gtrsim m_{Z}$ (Sec. III.2). Finally, some concluding remarks are presented in Sec. IV.

II The $U(1)_{B-L}^{(3)}$ model

We study a model where the SM gauge symmetries are extended to include $U(1)_{B-L}^{(3)}$ symmetry – the difference between the baryon number and the lepton number of the third generation. This symmetry is anomaly free, provided that we include a right-handed tau neutrino $\nu_{3R}$ to the SM. The charge assignments of the fermions are: $(Q_{3L},u_{3R},d_{3R}):1/3$ and $(\ell_{3L},e_{eR},\nu_{3R}):-1$, with all first and second generation fermions inert.

From various observations, we know the exact $U(1)_{B-L}^{(3)}$ symmetry is not realized in nature at low scales, and thus must be broken. The simplest mechanism to spontaneously break $U(1)_{B-L}^{(3)}$ is to add some scalars charged under $U(1)_{B-L}^{(3)}$ symmetry that acquire vacuum expectations values (vev). To make the model phenomenologically viable, we actually have to introduce two $U(1)_{B-L}^{(3)}$-charged scalars, a SM singlet $s$ with $U(1)_{B-L}^{(3)}$ charge $+1/3$ and $\phi$, a SM $SU(2)_{\text{W}}$ doublet with hypercharge $+1/2$ (identical SM charges as the Higgs) and $U(1)_{B-L}^{(3)}$ charge $+1/3$ 1705.01822 . The Table of particles charged under $U(1)_{B-L}^{(3)}$ is shown in Table 1.

The $\phi$ field is needed to connect first and second generation quarks to the third generation quarks via renormalizable interactions, while the additional source of $U(1)_{B-L}^{(3)}$ breaking from the $s$ field allows us to decouple the mass of the $U(1)_{B-L}^{(3)}$ gauge boson $X$ from the electroweak breaking scale. Note that Yukawa terms involving only third generation fields involve the Higgs, not $s$ or $\phi$, and that renormalizable inter-generation interactions involving the third generation between leptons are forbidden by the $U(1)_{B-L}^{(3)}$ charge assignment. Neutrino masses can be accommodated via higher dimensional operators or via further extensions of the model by vector-like matter 1705.01822 .

In this paper, we are interested in the phenomenology of the $X$ gauge boson. The $X$ gauge boson appears in the covariant derivative of the third generation fermions, indicating a tree-level interaction of $X$ with third generation of fermions in the interaction basis. Another place $X$ appears is the covariant derivative of scalars ($s$ and $\phi$), which not only results in $X$ acquiring a mass (once $\langle\phi\rangle,\langle s\rangle\neq 0$), but also leads to tree-level interactions of $X$ with scalars. Furthermore, because $\phi$ is charged under both $SU(2)_{\text{W}}\times U(1)_{\text{Y}}$ and $U(1)_{B-L}^{(3)}$, its kinetic term induces a mixing with $X$ and $Z$ gauge boson with an angle

[TABLE]

where $g_{X}$ is the gauge coupling associated with $U(1)_{B-L}^{(3)}$, $(v_{\phi})$ represents $\phi$ vev, and $v=\sqrt{v_{\phi}^{2}+v_{h}^{2}}=246\leavevmode\nobreak\ \mathrm{GeV}$, with $v_{h}$ being the Higgs vev. Therefore, in the mass basis, the (mass eigenstate) $X$ boson interacts with $Z$ current with a coupling proportional to $s_{X}$, while the (mass eigenstate) $Z$ boson interactions will be modified by an amount proportional to $s_{X}$.

In addition to $X$, the model contains several new scalars (from $\phi,s$) and a right-handed neutrino. For simplicity, and following Ref. 1705.01822 , we assume that these states are all heavier than $m_{X}/2$ so they play no role in our analysis.

The relevant model parameters to study $X_{\mu}$ phenomenology are the $X$ mass ($m_{X}$), the $U(1)_{B-L}^{(3)}$ gauge coupling ($g_{X}$), and the rotation angle between $Z$ and $X$ ($s_{X}$). Rather than use $s_{X}$, we find it more convenient to work with $\tan{\beta}=v_{h}/v_{\phi}$. In terms of these parameters,

[TABLE]

Notice that the presence of $v_{s}$ means $m_{X}$ is not tied to the electroweak scale and can, in principle, be large.

In the gauge interaction basis, the interaction between fermions and the (mass eigenstate) $X$ gauge boson has the form $c_{\alpha}\,\bar{f}_{\alpha}\gamma^{\mu}f_{\alpha}X^{\mu},$ with

[TABLE]

Here, $q_{\alpha}^{X}$ and $q_{\alpha}^{Z}=I_{3}^{\alpha}-s_{w}^{2}q_{\alpha}$ are respectively the $X$ and $Z$ charge of fermion $\alpha$, $q_{\alpha}$ is the electric charge, $s_{w}$ is the $\sin\theta_{\text{weak}}$, and $I_{3}^{\alpha}$ is the $W^{3}_{\mu}$ generator. The translation of this interaction to the fermion mass basis induces flavor changing interactions among left handed up-type quarks and is shown in detail in Appendix B.

An important property of $X$ for our study is how it decays to various SM states. Due to $X-Z$ mixing, the branching ratio of $X$ strongly depends on the value of $\tan{\beta}$. For small $\tan{\beta}$, the coupling of $X$ to the $Z$ current is important, while for large $\tan{\beta}$ $X$ predominantly decays to third generation fermions. The branching ratio of $X$ to various SM final states for $\tan{\beta}=1$ and $\tan{\beta}=5$ is shown below in Fig. 1 333There is a constraint on the value of $\tan{\beta}$ coming from Higgs coupling measurements, roughly between $1\,\leq\tan{\beta}\leq 30$. The exact range depends on the scalar $h,\phi,s$ spectrum, the details of which we ignore here, therefore we will work with this approximate range. . In this figure, we derived the branching ratio of $X$ to hadrons using Ref. Whalley:2003 , and we have assumed the new scalars and the sterile neutrino are heavier than $m_{X}/2$. We can see that the branching ratio of $X$ to a pair of $\tau$s dominates for $m_{X}\gtrsim 5\,\leavevmode\nobreak\ \mathrm{GeV}$ (and up to $m_{X}\sim 2m_{W}$). This channel dominates because of the relatively large values of $q_{\tau}^{X}$ and $q_{\tau}^{Z}$ compared to other third generation fermions.

Having defined the model, we now move on to its LHC signatures. As mentioned earlier, our focus is on $m_{X}>10\leavevmode\nobreak\ \mathrm{GeV}$ where low-energy constraints are absent. For $m_{X}>10\leavevmode\nobreak\ \mathrm{GeV}$ the only non-LHC bounds are from $\Upsilon\to\tau^{+}\tau^{-}$ and the modification of the oblique parameters. The contribution of off-shell $X$ to $\Upsilon$ decay dies off rapidly as $1/m_{X}^{2}$, and the constraints coming from oblique parameters become very mild for $\tan\beta>2$.

III LHC Constraints

The main advantage of the LHC is that it can produce the $X$ gauge boson on-shell. This fact is crucial, because amplitudes containing off-shell $X$ are suppressed by two powers of $g_{X}$ – one at the production vertex and another at its destruction. On-shell exchange, on the other hand, comes with only one factor of $g_{X}$ at the $X$ production vertex (amplitude level) as the decay portion contributes some $O(1)$ branching ratio factor.

The chief way to produce an on-shell $X$ at the LHC is in associated production with a pair of $b$ jets, $pp\to b\bar{b}X$. In such processes, we can benefit from the large QCD production of $b$s as well as the sizable coupling of $X$ with $b$ quarks. The $X$ boson can decay in many ways, however, we will focus here on $pp\to b\bar{b}X\to b\bar{b}\tau^{+}\tau^{-}$. This choice is motivated by the large $\text{Br}(X\to\tau^{+}\tau^{-})$, however there are some other important benefits:

• •

large number of observables to help signal background discrimination, in contrast to $b\bar{b}+\not{E}_{T}$ production.

• •

there are no combinatorics issues, as opposed to the $\bar{b}b\bar{b}b$ final state.

Because $\tau$s are not stable at the LHC, the search mode has to be further defined in terms of the $\tau^{+}\tau^{-}$ final decay products. While there exists several options, we find that requiring at least one of the $\tau$s to decay leptonically is necessary to suppress the (otherwise enormous) QCD background. To decide between semi-leptonic $\tau$s or fully leptonic ones, let us turn our attention to potential triggers444As we will show in the subsequent sections, the $E_{T}{/}\hskip 9.95863pt$distribution in the signal favors lower values. Therefore, a $E_{T}{/}\hskip 9.95863pt$trigger is also not ideal for our study. As leptons are relatively clean objects at the LHC, they have softer trigger cuts, and the presence of multiple leptons softens the requirement on each lepton further. As an example, the single lepton trigger at CMS  Khachatryan:2016fll ; 1611.04040 requires $p_{T}(\ell)>23\ (20)\leavevmode\nobreak\ \mathrm{GeV}$ if the lepton is electron (muon), and the CMS di-lepton trigger Khachatryan:2016fll ; 1611.04040 requires $p_{T}(\ell_{1})>17\leavevmode\nobreak\ \mathrm{GeV}$ and $p_{T}(\ell_{2})>12\ (8)\leavevmode\nobreak\ \mathrm{GeV}$ with the second leading lepton being electron (muon)555The ATLAS numbers trigger cuts are similar: single lepton requires the $p_{T}(\ell)>26\leavevmode\nobreak\ \mathrm{GeV}$ Aaboud:2017ojs ; Aaboud:2017buh , while the di-lepton trigger requires $p_{T}(\ell)>17\leavevmode\nobreak\ \mathrm{GeV}$, on both of the leading leptons Aaboud:2017buh .. Production of $b\bar{b}X$ is dominated near threshold (rather than with boosted $X$), hence the leptons coming from a light $X$ are expected to have small $p_{T}$. Therefore, for low $m_{X}$ the fully leptonic $\tau$s is the better option since the softer thresholds in the dilepton trigger will accept more signal. For high $m_{X}$, the leptons from $X$ decay are significantly energetic to be picked up efficiently by the single lepton trigger. This makes the semi-leptonic mode viable, and its larger branching fraction (compared to dileptonic taus) partially compensates for the drop in the signal cross section as $m_{X}$ increases.

As the optimal $b\bar{b}\tau^{+}\tau^{-}$ final state depends on $m_{X}$, we divide our analysis to two sections. In the following section (Sec. III.1), we study $X$ gauge bosons with $m_{X}<m_{Z}$ using the $b\bar{b}\ell^{+}\ell^{-}\not{E}_{T}$ final state. Then, in Sec. III.2, we use the $b\bar{b}\ell j\not{E}_{T}$ final state to explore heavier $X$ gauge bosons, $m_{X}\sim[m_{Z},2\,m_{W})$. To thoroughly study the LHC detection prospects, we generated a Universal FeynRules Output (UFO) model Degrande:2011ua using Feynrules Alloul:2013bka . We then fed the model to MadGraph-aMC@NLO Alwall:2014hca ; Alwall:2011uj for all simulations666We used the nn23lo1 parton distribution functions for all event samples, with factorization scale and renormalization scale set to their default value, $\hat{s}$., including the calculation of $X$ total width for a given coupling ($g_{X}$ and $s_{X}$). We used Pythia 8.2  Sjostrand:2014zea for hadronization, showering, and $\tau$ decay, and Delphes deFavereau:2013fsa with default cards for detector smearing, flavor tagging and jet reconstruction.

III.1 Light $X$: $m_{X}<m_{Z}$

As discussed above, for $m_{X}<m_{Z}$ the final state we are interested in extracting is $b\bar{b}\ell^{+}\ell^{-}\not{E}_{T}$. The main contribution to the signal comes from $pp\to b\bar{b}X$ with $X\to\tau^{+}\tau^{-}$ (specifically, the process $pp\to\tau^{+}\tau^{-}X$ followed by $X\to b\bar{b}$ only contributes to the signal at sub-percent level); Fig. 2 shows some of the signal processes. Because $X$ is predominantly produced on-shell, the signal has little interference with the SM backgrounds. Therefore, to a good approximation, the cross section can be expressed as:

[TABLE]

where $c_{b}(g_{X},\tan{\beta})$ is the coupling of $X$ with b quarks. The subscript $\tan{\beta}$ in Eq. (4), indicates that the branching ratio of $X$ depends on $\tan\beta$. Technically, the branching ratio depends on $m_{X}$ as well, however, for the mass range of $2m_{b}\leq m_{X}<2m_{{}_{W}}$ the branching ratios are constant with respect to $m_{X}$. The remaining part of the cross section, $f(m_{X},s)$, governs the kinematics and is a function of $m_{X}$ and collider energy $s$ only. In our simulations, we generated events for $m_{X}=10,\ 20,\ 30,\ 50,$ and $80\leavevmode\nobreak\ \mathrm{GeV}$, and fixed $g_{X}=0.02$ and $\tan{\beta}=5$. However, as $c_{b}(g_{X},\tan{\beta})\times\text{Br}(X\to\tau^{+}\tau^{-})_{\tan{\beta}}$ do not play a noticeable role in the kinematic distributions (which govern signal acceptances), the LHC sensitivity at one $g_{X},\tan{\beta}$ value can be extended to other $g_{X},\tan{\beta}$ values simply by rescaling:

[TABLE]

where the indices $n$ and $o$ refer to new and old, respectively.

There are a number of SM process that give rise to the $b\bar{b}\ell^{+}\ell^{-}\not{E}_{T}$ final state, namely:

[TABLE]

where $\ell=e,\mu$, and $(W/Z)_{\ell+\tau}$ refers to all possible charged leptonic decay of $W/Z$. Similarly, $\tau_{\ell}$ refers to the decay of $\tau$ to lighter leptons. The main difference between these three backgrounds is the number of neutrinos. The first background has two or six neutrinos, depending on whether the gauge bosons decay to $\ell$ or $\tau_{\ell}$, respectively, the second background has four, and the third does not have any neutrinos. However, due to pile-up, jet mis-measurement, and the leptonic decays of a charged $B$ mesons in the b-jets, a net $E_{T}{/}\hskip 9.95863pt$can be generated, making the last background worthy of mentioning.

A few of the important Feynman diagrams for the SM production of $b\bar{b}\ell^{+}\ell^{-}\not{E}_{T}$ are shown in Fig. 3. The largest irreducible background comes from $pp\to t\bar{t}\to b\bar{b}W^{+}W^{-}$, as it is purely a QCD process with $\text{Br}(t\to bW)\simeq 1$. Other (continuum) processes contributing to the first background in Eq. (10) do not have a large rate. The second background, $b\bar{b}\tau^{+}_{\ell}\tau^{-}_{\ell}$, is very similar to the signal, where $X$ is replaced with a $Z$ or $\gamma$. Technically, the $\left(pp\to b\bar{b}\tau^{+}_{\ell}\tau^{-}_{\ell}\right)$ background and the signal can interfere, however the fact that $X$ are predominantly produced on-shell renders the interference is very small777 In our simulations, we force $X$ to be on-shell. To make sure this shortcut does not significantly influence our results, we tested the effects of off-shell $X$ (and interference) for various values of $m_{X}$. In all cases, the difference between on-shell $X$ and the full treatment was negligible.. The last background is similar to the second background, but instead of producing a pair of $\tau$ leptons, $Z/\gamma\to\ell^{+}\ell^{-}$.

We must also include (reducible) backgrounds where a gluon/light flavor jet is mis-identified as a b-jet. The mis-identification rate of a $c$ jets is significantly higher than other light quark/gluon initiated jets (collectively referred to as $j_{l}$). Therefore, we considered charm-jet and light-jet processes separately. To include the impact of misidentifications, we add two versions of all backgrounds in Eq. (10) – one with $b$ jets replaced with $c$ and one with $b$ replaced by $j_{l}$. For example, the second background is expanded to include:

[TABLE]

Other backgrounds induced by lepton mis-identification are expected to have a very low rates Khachatryan:2015hwa ; Aaboud:2016vfy ; Sirunyan:2018fpa ; Aad:2016jkr and thus are ignored in this study. The MC event samples for all processes are simulated at leading order (LO), with the overall rates scaled to next-to-leading-order (NLO)888Using MadGraph5-aMC@NLO for a 13 TeV LHC, we find the $\kappa_{b\bar{b}V_{\ell+\tau_{\ell}}V_{\ell+\tau_{\ell}}}\equiv\sigma_{\rm NLO}/\sigma_{\rm LO}$ of the $b\bar{b}V_{\ell+\tau_{\ell}}V_{\ell+\tau_{\ell}}$ process is roughly $1.4$, 1.8 for $\kappa_{b\bar{b}\tau^{+}\tau^{-}}$, 1.9 for $\kappa_{b\bar{b}\ell^{+}\ell^{-}}$. For the reducible background where a light flavor/gluon jet is mis-identified as a b-jet, we assume $\kappa=2$. The $\kappa$ of the signal is assumed to be $1.8$, due to the similar topology of the signal with the $b\bar{b}\tau^{+}\tau^{-}$ background. .

Before studying the kinematic distributions, we impose some preliminary cuts to ensure the events have been triggered upon, and that the visible final states are within the fiducial region of the detector. Specifically, we select events that satisfy the following requirements:

• – Include exactly two isolated opposite sign leptons (any combination of electrons and muons) with $|\eta|<2.5$ and separated from each other by $\Delta R>0.4$. We further require the leptons to satisfy the dilepton trigger: the leading lepton must have $p_{T}(\ell_{1})>17\,\leavevmode\nobreak\ \mathrm{GeV}$, and the second leading lepton is required to have $p_{T}(\ell_{2})>12(8)\,\leavevmode\nobreak\ \mathrm{GeV}$ if the lepton is electron (muon).
• – Include exactly two jets with $p_{T}(j)>30\leavevmode\nobreak\ \mathrm{GeV}$, $|\eta|<2.5$, and separated from each other by $\Delta R>0.4$. Both jets must be $b$-tagged. We use the b-tagging option in Delphes deFavereau:2013fsa , which corresponds to roughly a b-tagging efficiency of $60\%$, with a charm mistagging of $15\%$ and a light jet mis identification rate of $1\%$.

After these requirements, the largest background is $b\bar{b}V_{\ell+\tau_{\ell}}V_{\ell+\tau_{\ell}}$ with cross section of $1362\,\leavevmode\nobreak\ \mathrm{fb}$, followed by $b\bar{b}\ell^{+}\ell^{-}$ with $255\,\leavevmode\nobreak\ \mathrm{fb}$ cross section, while that of $b\bar{b}\tau^{+}\tau^{-}$ is $28\leavevmode\nobreak\ \mathrm{fb}$. Among the reducible (fake $b$) backgrounds, $c\bar{c}\ell^{+}\ell^{-}$ ($20.3\,\leavevmode\nobreak\ \mathrm{fb}$), $j_{l}j_{l}\ell^{+}\ell^{-}$ ($8\,\leavevmode\nobreak\ \mathrm{fb}$), and $c\bar{c}\tau^{+}\tau^{-}$ ($2.4\,\leavevmode\nobreak\ \mathrm{fb}$) are the most significant. The rest of the irreducible backgrounds have cross sections $\leq 2\,\leavevmode\nobreak\ \mathrm{fb}$ (see Table 2).

Thankfully, the topology of the dominant background – $b\bar{b}V_{\ell+\tau_{\ell}}V_{\ell+\tau_{\ell}}$ – is vastly different from the signal, giving us a hope to reduce it further with additional kinematic cuts. One variable that is particularly useful in teasing out the signal is the invariant mass of the leptons. In the signal we know $m_{\ell\ell}<m_{X}$, while the invariant mass of the leptons in $b\bar{b}V_{\ell+\tau_{\ell}}V_{\ell+\tau_{\ell}}$ has a broad, featureless distribution. Therefore, requiring an upper bound on $m_{\ell\ell}$ can significantly suppress the $b\bar{b}V_{\ell+\tau_{\ell}}V_{\ell+\tau_{\ell}}$ background while retaining most of the signal region. A comparison of the $m_{\ell\ell}$ distribution (area normalized) for the background and a signal benchmark, $m_{X}=30\leavevmode\nobreak\ \mathrm{GeV}$ is shown below in the left panel of Fig. 4.

For light $m_{X}$, the only background that behaves similarly to the signal is $pp\to\gamma^{*}\to\tau^{+}_{\ell}\tau^{-}_{\ell},\ell^{+}\ell^{-}$. The cross section of this background is highly suppressed by the isolation cut $\Delta R(\ell^{+},\ell^{-})>0.4$. The isolation cut also impacts the signal for small values of $m_{X}$, however, for all benchmarks we are considering ($m_{X}\geq 10\,\leavevmode\nobreak\ \mathrm{GeV}$) imposing lepton isolation is more beneficial than relaxing it.

To understand the discriminatory power of the $m_{\ell\ell}$ cut, we plot the $m_{\ell\ell}$ of backgrounds normalized to actual cross section, in Fig. 5. The dark blue region is the distribution of the signal with $m_{X}=30\leavevmode\nobreak\ \mathrm{GeV}$. The cyan region belongs to the $b\bar{b}V_{\ell+\tau_{\ell}}V_{\ell+\tau_{\ell}}$ distribution, the red shaded region is the $b\bar{b}(\ell^{+}\ell^{-})_{Z}$, and the magenta dotted distribution belongs to $b\bar{b}(\ell^{+}\ell^{-})_{\gamma}$ distribution. The reducible background $c\bar{c}\ell^{+}\ell^{-}$ and $j_{l}j_{l}\ell^{+}\ell^{-}$ are also shown in red. The smooth green region belongs to $b\bar{b}\tau_{\ell}^{+}\tau^{-}_{\ell}$, and the dotted green distribution is for $c\bar{c}\ \tau^{+}_{\ell}\tau^{-}_{\ell}$. Only basic selections have been imposed on these distributions.

For our $m_{X}=30\leavevmode\nobreak\ \mathrm{GeV}$ benchmark, imposing $m_{\ell\ell}<25\leavevmode\nobreak\ \mathrm{GeV}$ eliminates 96% of the background while retaining $97\%$ of the signal. For other $m_{X}$ benchmarks, an appropriately optimized $m_{\ell\ell}$ cut performs similarly, though its effectiveness decreases for larger $m_{X}$. The decrease can be understood by looking at the $m_{\ell\ell}$ distribution for various $m_{X}$, shown in the right panel of Fig. 4. As we increase $m_{X}$, the $m_{\ell\ell}$ distribution broadens and gets more separated from the $m_{X}$ value. The broadening occurs as a result of the allocation of the $X$’s energy between leptons and neutrinos. The power of $m_{\ell\ell}$ in distinguishing the signal for each of the benchmarks is tabulated in Appendix A, where the upper bound on $m_{\ell\ell}$ (approximately $m_{\ell\ell}\lesssim 4/5m_{X}$) has been optimized for each $m_{X}$ value.

Another variable that is useful in signal-background discrimination is the azimuthal angle between the dilepton and the net missing energy vector $\Delta\phi(\ell\ell,\not{E}_{T})$. Since leptons and neutrinos come from $X$, we expect to see some angular correlations between the leptons and the $E_{T}{/}\hskip 9.95863pt$vector. Because we do not know the pseudo-rapidity of the transverse missing energy ($E_{T}{/}\hskip 9.95863pt$) vector, the distribution of azimuthal angle provides a better discrimination than the total separation. A comparison between the two distributions of $\Delta R(\ell\ell,\not{E}_{T})$ and $\Delta\phi(\ell\ell,\not{E}_{T})$ is presented in Fig 6. For the benchmarks with $m_{X}\leq 50\leavevmode\nobreak\ \mathrm{GeV}$, the optimum cut seems to be $\Delta\phi(\ell\ell,\not{E}_{T})\lesssim 0.5$.

Finally, we use the $E_{T}{/}\hskip 9.95863pt$distribution to further discriminate between the signal and the background. As shown in Fig. 7, the signal favors low $E_{T}{/}\hskip 9.95863pt$regime. That is because $b\bar{b}X$ production is maximum near threshold, where $X$ is almost stationary. The two neutrinos are, therefore, almost back-to-back, resulting in low $E_{T}{/}\hskip 9.95863pt$in the signal. Thereby, we can impose an upper limit on $E_{T}{/}\hskip 9.95863pt$, to suppress $b\bar{b}V_{\ell+\tau_{\ell}}V_{\ell+\tau_{\ell}}$ and $b\bar{b}\tau^{+}_{\ell}\tau^{-}_{\ell}$ backgrounds.

Unfortunately, one background that favors low $E_{T}{/}\hskip 9.95863pt$is $b\bar{b}\ell^{+}\ell^{-}$, because its $E_{T}{/}\hskip 9.95863pt$is mostly a result of mismeasurement. To reduce this background, we must impose a lower bound on $E_{T}{/}\hskip 9.95863pt$in addition to the upper bound. The exact upper and lower $E_{T}{/}\hskip 9.95863pt$cuts were determined using our simulated events and adjusted to optimize the significance; the specific values for each of the benchmarks is presented in Appendix A, however the cut on $E_{T}{/}\hskip 9.95863pt$is roughly $10\lesssim\not{E}_{T}\lesssim 70\leavevmode\nobreak\ \mathrm{GeV}$.

To quantify the sensitivity of our search, we follow the conventional definitions:

[TABLE]

In other words, $S$ and $B$ are respectively the number of signal and total background events (scaled to NLO rates) that we expect to observe at the LHC for a given luminosity and our cut flow. Using these, we quantify the discovery potential of our analysis using the significance, defined –following Ref. 1404.0682 – as:

[TABLE]

where $\lambda_{i}$ represent the systematic uncertainties associated with each background. We used the following values for $\lambda_{i}$, taken from the CMS leptonic $b\bar{b}V_{\ell+\tau_{\ell}}V_{\ell+\tau_{\ell}}$ search 1603.02303 ; Aaboud:2016pbd : $\lambda_{b\bar{b}V_{\ell+\tau_{\ell}}V_{\ell+\tau_{\ell}}}=5\%$, $\lambda_{b\bar{b}\tau^{+}\tau^{-}}=10\%$, and $\lambda_{b\bar{b}\ell^{+}\ell^{-}}=15\%$. Finally, it is important to note that we have ignored the effect of pile-up in our analysis. Including the effect of pile-up will likely affect the lower bound on $E_{T}{/}\hskip 9.95863pt$. Specifically, it will effect the contribution of the $b\bar{b}\ell^{+}\ell^{-}$ process in the background. However, the cross section of $b\bar{b}\ell^{+}\ell^{-}$ even before imposing the $E_{T}{/}\hskip 9.95863pt$cut is already much smaller than other processes, and thus we do not expect that a small change in its cross section to alter our results significantly.

For each $m_{X}$ benchmark, the cuts on the $m_{\ell\ell},\Delta\phi(\ell\ell,\not{E}_{T})$ and $\not{E}_{T}$ distributions have been optimized to yield the largest significance (Eq. (15)). Once the cuts have been optimized, we use Eq. (5) to extrapolate the analysis to other $g_{X}$ values and trace out contours of a desired significance, as shown in Fig. 8. The red lines are the bounds with (roughly) the current luminosity of the LHC – $100\leavevmode\nobreak\ \mathrm{fb}^{-1}$ – while the green lines are the projected sensitivity with the full luminosity of HL-LHC ($3\,\leavevmode\nobreak\ \mathrm{ab}^{-1}$). The solid lines present the 3 $\sigma$ (significance as defined in Eq. (15) = 3) exclusion bound assuming the full systematical uncertainties mentioned earlier, and the dashed lines are $3\ \sigma$ significance when the systematic uncertainties are cut into half.

As we can see from Fig. 8, the LHC can probe a region of the parameter space that is out of reach of other current experiments. The LHC bounds are best in the mass range $20-40\leavevmode\nobreak\ \mathrm{GeV}$. The constraints on lighter $X$ are milder, due to the low dilepton trigger efficiency and a relatively lower b-tagging efficiency, and the sensitivity for heavier $X$ drops because the distribution of the signal and backgrounds become more similar, and so the cuts become less efficient.

Above $50\,\leavevmode\nobreak\ \mathrm{GeV}$, the limits worsen quickly. Therefore, it is worth exploring if there are any additional variables that can improve the bounds in the mass range $50\leavevmode\nobreak\ \mathrm{GeV}<m_{X}<m_{Z}$. The most troublesome background in this mass range is $b\bar{b}V_{\ell+\tau_{\ell}}V_{\ell+\tau_{\ell}}$, and the most important contribution to $b\bar{b}V_{\ell+\tau_{\ell}}V_{\ell+\tau_{\ell}}$ comes from $t\bar{t}$. The goal of the next sub-section is to investigate some kinematic handles that specifically target the $t\bar{t}$ background.

III.1.1 Further Separation of the Signal from the $b\bar{b}V_{\ell+\tau_{\ell}}V_{\ell+\tau_{\ell}}$ background, for $m_{X}\sim m_{Z}$

The dominant background to our signal is $t\bar{t}\to b\bar{b}W^{+}_{\ell,\tau_{\ell}}W^{-}_{\ell,\tau_{\ell}}$. This background has some specific features that can help us separate this process from the signal. For example, the leptons and neutrinos in $t\bar{t}$ come from $W$ decays, whereas in the signal, they are due to $\tau$ decays. Therefore, $M_{T2}$, defined as

[TABLE]

with $\nu^{i}_{T}$ being the transverse momentum of the either sources of missing energy, should show a decent separation between the signal and the $t\bar{t}$ background. Since we are particularly interested in enhancing the sensitivity for $m_{X}\sim m_{Z}$, in the left panel of Fig. 9, we compare the distribution of the benchmark with $m_{X}=80\,\leavevmode\nobreak\ \mathrm{GeV}$ with the backgrounds. As expected, $M_{T2}$ of the signal prefers small values ($\lesssim 20\leavevmode\nobreak\ \mathrm{GeV}$), while $M_{T2}$ for the $t\bar{t}$ background $\sim m_{W}$. The distribution of $M_{T2}$ for various benchmarks, after basic cuts only (so no $m_{\ell\ell},\Delta\phi(\ell\ell,\not{E}_{T})$, etc.), is also shown in the right panel of Fig. 9. Even though the separation of the signal from the $b\bar{b}V_{\ell+\tau_{\ell}}V_{\ell+\tau_{\ell}}$ background is more visible for lighter $X$, $M_{T2}$ does not improve the bounds for $m_{X}<50\leavevmode\nobreak\ \mathrm{GeV}$ compared to the combination of kinematic cuts introduced in the previous section. For heavy $X$, however, the efficiency of a cut on $\Delta\phi(\ell\ell,\not{E}_{T})$ and $\not{E}_{T}$ distributions is not as efficient as a cut on $M_{T2}$. By requiring $M_{T2}<16\leavevmode\nobreak\ \mathrm{GeV}$, the significance goes up by a factor of 3 for $m_{X}=80\,\leavevmode\nobreak\ \mathrm{GeV}$, and the mild discrimination in the $\Delta\phi(\ell\ell,\not{E}_{T})$ and $\not{E}_{T}$ distributions fade off. Therefore, we can no longer impose an efficient cut on $\Delta\phi(\ell\ell,\not{E}_{T})$ and $\not{E}_{T}$ distributions.

Another attribute of the $t\bar{t}$ background is that there is an intimate relationship between $b$ jets and leptons: $m_{bl}\lesssim m_{t}$. In the signal, on the other hand, such correlation does not exist, and $m_{bl}$ can take any arbitrary values. To take advantage of this difference, we study $m_{b\ell}$ in Fig. 10, where the $b-\ell$ is one of the combinations that minimizes $(m_{\ell_{i}b_{k}}-m_{t})^{2}+(m_{\ell_{j}b_{n}}-m_{t})^{2})$, with $i\neq j$ and $k\neq n$. The distributions for $m_{b\ell}$ for the backgrounds and a $m_{X}=80\,\leavevmode\nobreak\ \mathrm{GeV}$ benchmark are shown below in Fig. 10. We can see that there is a modest separation999According to the distributions in Fig. 10, the signal mostly resides in $m_{b\ell}\lesssim 200\leavevmode\nobreak\ \mathrm{GeV}$, and thus the CMS search for third generation leptoquarks CMS:2018pab ; Khachatryan:2014ura with $m_{b\ell}>250\leavevmode\nobreak\ \mathrm{GeV}$ does not have a noticeable sensitivity to our signal. between the $t\bar{t}$ background and other processes. A cut on this distribution can enhance the significance by $15\%$ for the benchmark $(m_{X},g_{X})=(80\leavevmode\nobreak\ \mathrm{GeV},0.02)$.

To improve on this guess, we tried finding the neutrino momenta by reconstructing the $W$ and top mass. In particular, we scanned through all possible values of momenta that give the smallest $\chi^{2}$, defined as:

[TABLE]

where $\sigma_{W}$ and $\sigma_{t}$ are arbitrary values we can use to enhance our discrimination. However, regardless of the values of $\sigma_{t,W}$, this method did not improve the signal-background discrimination. Therefore, the only cuts that could improve our sensitivity to $m_{X}\sim m_{Z}$ were $M_{T2}<16\leavevmode\nobreak\ \mathrm{GeV}$ and $m_{bl}<100\leavevmode\nobreak\ \mathrm{GeV}$. With these cuts, more than $94\%$ of the background is removed, while almost $50\%$ of the signal is preserved. The effect of these cuts on the significance is shown in Fig. 11. Since the cross section is proportional to two factors of the coupling ($\sigma\propto g_{X}^{2}$), improving the limit by a factor of 4 translates into an improvement in the coupling by a factor of 2.

Having exhausted the cut-based search strategies for light $m_{X}$, we now turn to $m_{X}>m_{Z}$. In this regime, the $\tau$s coming from the decay of $X$ gauge boson are expected to be energetic enough such that the resulting lepton from one of the $\tau$s can pass the single lepton trigger with high efficiency. Therefore, we study the semi-leptonic $b\bar{b}\tau_{h}\tau_{\ell}$.

III.2 Heavy $X$: $m_{X}>m_{Z}$

The cross section for $X$ production falls as $m_{X}$ increases. To compensate for the lower cross section, for $m_{X}>m_{Z}$ we shift final states to semi-leptonic $\tau$s (one tau decays to leptons, one to hadrons) to take advantage of the higher branching ratio of $\tau$ to hadrons. The SM backgrounds we need to be concerned about are

[TABLE]

where, $\ell=e,\mu$, and $\tau_{\ell}$ refers to the leptonic decay of a $\tau$. Similarly, $h$ represents the hadronic decay. Only the first two backgrounds mentioned in Eq. 18 are irreducible. As hadronic taus can be faked by ‘normal’ jets ($=j_{l}+c+\bar{c}+b+\bar{b}$), we must include lepton + jet backgrounds such as 3.) and 4.) above101010Backgrounds 3.) and 4.) in Eq. (18) are separated as they contain different numbers of charged leptons. The third background has one lepton, and the fourth background contains two leptons, with some probability that one of the leptons fall outside of the acceptance and manifests itself as missing energy. . To estimate the backgrounds with fake taus, we rely on the built-in tau identification algorithm in Delphes, where we input matched samples111111We use MLM matching with Madgraph5 + PYTHIA8, with ${\tt xqcut}=20$ Alwall:2007fs . We have included up to $2$ jets, e.g $pp\rightarrow\,b\bar{b}W^{\pm}_{\ell+\tau_{\ell}}+0,1,2\,\text{jets}$. The matched/merged cross sections are then rescaled to the $+0\,\text{jet}$ NLO values..

As in the previous section, we must also consider backgrounds where a charm jet or a light quarks/gluon jet ($j_{l}$) is mis-identified as a b-jet. For example:

[TABLE]

where ‘$+\,\text{jets}$’ is treated as up to $+0-2$ jets.

To capture the interesting events, we impose the following conditions:

• – Each event must include exactly one charged lepton that passes the single lepton trigger: $p_{T}(\ell_{1})>27\ (24)\,\leavevmode\nobreak\ \mathrm{GeV}$ if the lepton is electron (muon). We also require $|\eta(\ell)|<2.5$.
• – We require one $b$-jet possessing $p_{T}(j)>50\,\leavevmode\nobreak\ \mathrm{GeV}$, and $|\eta(j)|<2.5$. As in the previous section, we use the Delphes deFavereau:2013fsa b-identification algorithm to tag a $b$ jet.
• – Every event must contain one tau-tagged hadronic jet, $p_{T}(j)>50\,\leavevmode\nobreak\ \mathrm{GeV}$, and $|\eta(j)|<2.5$. As with $b$-jets, we rely on the built-in algorithm in Delphes deFavereau:2013fsa . We find the $\tau$ tagging efficiency is roughly $40\%$ for correctly identifying a hadronic $\tau$ with an $\sim 0.3\%$ risk of misidentifying a normal jet as a hadronic $\tau$, for the processes being considered here.
• – In addition to the $b$-jet and $\tau$ jet, the event may contain at most one additional jet, $p_{T}(j)>50\,\leavevmode\nobreak\ \mathrm{GeV}$, and $|\eta(j)|<2.5$. The separation between each jet, as well as the separation between all jets and the lepton must be greater than $0.4$.

Due to the presence of multiple jets in our final state of interest, one might expect the main backgrounds come from tau/b-misidentified jets. However, after requiring the basic cuts mentioned, the largest background is the irreducible $b\bar{b}\ W_{\tau_{{}_{h}}}W^{\pm}_{\ell+\tau_{{}_{\ell}}}$, 120 fb at NLO. The other sizable backgrounds are $b\bar{b}\ \tau_{h}\tau^{\pm}_{\ell}$, (3.5 fb), $b\bar{b}W^{\pm}_{\ell+\tau_{\ell}}+\text{jets}$ (3 fb), and $b\bar{b}\ (Z/\gamma)_{\ell+\tau_{\ell}}+\text{jets}$ (0.3 fb). All other backgrounds are negligible, $\ll 0.1\leavevmode\nobreak\ \mathrm{fb}$.

To enhance the sensitivity of the signal further, we studied various kinematic distributions including $M_{T2}$, $m_{\ell j_{i}}$ – where $j_{i}$ is any of the jets in the final state, the separation between the lepton and the jets $\Delta R(\ell,j_{i})$, the difference in the azimuthal angle between any two visible objects in the final state, as well as the $p_{T}$ of each of the visible final states. Some of these distributions show a small difference between the signal and background, but none of them have a considerable effect on their own. Therefore, for this initial study, we will ignore the impact of these other distributions and quantify the sensitivity using the basic cuts alone. A multivariate analysis may be able to harness the slight differences across several kinematic distributions and yield increased sensitivity. Such an approach would be interesting to pursue, but is beyond the scope of the current work. However, as the difference in the distributions are very small, we expect the sensitivity gains achieved by a MVA to be $\mathcal{O}(1)$ in the cross section and not orders of magnitude.

Using a similar definition of the signal and background as in Section III.1:

[TABLE]

with

[TABLE]

and extrapolating to all values of $g_{X},\tan{\beta}$ using Eq. (5), we can chart significance contours. In the $m_{X}>m_{Z}$ region, the main background is the irreducible background $pp\to t\bar{t}\to b\bar{b}\ W_{\tau_{{}_{h}}}W^{\pm}_{\ell+\tau_{{}_{\ell}}}$, which has a systematic uncertainty of $\lambda=12\%$ Aaboud:2018mjh . We will assume the same uncertainty on the rest of the backgrounds as well, though an $O(1)$ change in the systematic uncertainties of other backgrounds does not affect the results significantly. Assuming $100\leavevmode\nobreak\ \mathrm{fb}^{-1}$ integrated luminosity, we can exclude up to $g_{X}\gtrsim 0.07$ for $m_{X}=100\leavevmode\nobreak\ \mathrm{GeV}$, and $g_{X}\gtrsim 0.2$ for $m_{X}=150\leavevmode\nobreak\ \mathrm{GeV}$ up to $3\sigma$ significance, illustrated in Fig. 12. This is about a factor of $2-4$ improvement over previous constraints. Even though the total background of the semi-leptonic $b\bar{b}\tau_{h}\tau_{\ell}$ after the basic cuts is much smaller than that of the fully leptonic $b\bar{b}\tau_{\ell}\tau_{\ell}$, the constraints in the $m_{X}\ll m_{Z}$ region are much stronger. That is because in the $m_{X}\ll m_{Z}$ region, the kinematic distributions of the signal have sharp features that distinguishes it from the background. For larger masses, however, the distributions broaden and lose their sharp features and thus separating the signal from the background is more challenging.

In general, a dedicated search at the LHC can improve the bounds by a factor of $2-10$. These results can be achieved by studying simple kinematic distributions. With the usage of a more advanced technique like an MVA, we might obtain even better results. Moreover, we have stopped our search at $m_{X}<2m_{W}$. The bounds on a larger $m_{X}$ will depend on some parameters in the scalar potential that we have ignored for our study (e.g, mixing between the scalars). If $m_{X}>2m_{t}$, the decay of $X$ to a pair of top quarks enjoys a significant probability as well as small background due to the large number of final state particles. These searches have already received some attention in several phenomenological studies Hill:1991at ; Hill:1993hs ; Hill:1994hp ; Harris:2011ez ; Rosner:1996eb ; Lynch:2000md ; Carena:2004xs ; Choudhury:2007ux ; Khachatryan:2015sma ; CMS:2018ohu ; Aaboud:2018mjh ; Sirunyan:2017uhk ; Sirunyan:2017yar ; Cerrito:2016qig ; Arina:2016cqj ; Pedersen:2015knf ; Fox:2018ldq . The constraints obtained by these studies can be recasted according to our choice of model parameter values.

IV Conclusion

In this work, we explored the LHC potential to constrain $X$, the gauge boson associated with a spontaneously broken $U(1)_{B-L}^{(3)}$ symmetry. $U(1)_{B-L}^{(3)}$ symmetry is one of the simplest extensions of the SM, which was first proposed to explain the flavor alignment of the third generation of quarks. While $X$ only interacts with the third generation of fermions in the interaction basis (at tree-level), flavor non-universal couplings are generated once we rotate to the mass basis. These flavor-violating effects can be mitigated with certain charge assignments and coupling assumptions, but strong constrains from low-energy experiments persist for $m_{X}\leq 5\,\leavevmode\nobreak\ \mathrm{GeV}$.

To hunt for heavier $X$, which are free from low-energy bounds, we developed two dedicated LHC search strategies based on $pp\to X,X\to\tau^{+}\tau^{-}$, a production and decay path that yields a high rate and numerous kinematic handles to suppress SM backgrounds. Following Ref. 1705.01822 , we assume all scalars related to $U(1)_{B-L}^{(3)}$ breaking and the right handed neutrino are heavy, and focus on $m_{X}\leq 2\,m_{W}$ since this decouples the $X$ phenomenology from any mixing in the scalar sector.

For $m_{X}<m_{Z}$, we find the optimal channel is where both taus decay leptonically. Using a combination of simple kinematic variables, such as $m_{\ell\ell},\Delta\phi(ll,\not{E}_{T})$ and $E_{T}{/}\hskip 9.95863pt$, we find that couplings as low as $g_{X}>0.005$ for $m_{X}\sim 20\leavevmode\nobreak\ \mathrm{GeV}$ could be probed at $3\ \sigma$ sensitivity given $100\leavevmode\nobreak\ \mathrm{fb}^{-1}$ integrated luminosity (roughly the current total LHC-13 luminosity). For heavier masses, the bounds are not as strong: $g_{X}>0.05$ for $m_{X}\sim 80\leavevmode\nobreak\ \mathrm{GeV}$ probed at $3\,\sigma$ with the same amount of data. Extrapolating these bounds to the full HL-LHC luminosity of $3\,\text{ab}^{-1}$, we expect a further increase by a factor of 2 in the sensitivity (or $\sqrt{2}$ in $g_{X}$).

For $m_{X}>m_{Z}$, we find the semi-leptonic tau channel ($\tau^{+}_{h}\tau^{-}_{\ell}$ + $\tau^{+}_{\ell}\tau^{-}_{h}$) outperforms the fully leptonic mode, however the number of pronounced kinematic differences between the signal and the dominant background ($pp\to t\bar{t}$) shrinks substantially. For $100\leavevmode\nobreak\ \mathrm{fb}^{-1}$, we find the $3\,\sigma$ exclusion limit reaches $g_{X}\gtrsim 0.1$ for $m_{X}=100\leavevmode\nobreak\ \mathrm{GeV}$, and $g_{X}\gtrsim 0.2$ for $m_{X}=150\leavevmode\nobreak\ \mathrm{GeV}$. Both the low-mass and high-mass search strategies relied on cut-and-count methods and it would be interesting to explore what improvements multivariate techniques can squeeze out.

Acknowledgments

We are particularly thankful to S. Chenarani for the numerous insightful conversations. We would like to also thank H. Hesari, S. Khatibi, J. H. Kim, M. Paktinat, and F. Rezai for their useful comments. We are grateful to CERN theory group for their hospitality. The work of AM was partially supported by the National Science Foundation under Grants No. PHY-1820860.

Appendix A The cut flow of benchmarks with $m_{X}<m_{X}$

The cut flow on each of the benchmarks is shown here. The quoted cross sections are at NLO, even though the events are LO. We generated $10^{6}$ events for the signal, $b\bar{b}\ell^{+}\ell^{-}$, and $b\bar{b}\tau_{\ell}^{+}\tau_{\ell}^{-}$ processes. Due to the higher cross section of $b\bar{b}V_{\ell+\tau_{\ell}}V_{\ell+\tau_{\ell}}$, we generated $2\times 10^{6}$ events for this process. In all of the benchmarks studied here, $m_{\ell\ell}$ proved to be a useful variable in distinguishing the signal. For $m_{X}\leq 50\leavevmode\nobreak\ \mathrm{GeV}$, we used $\Delta\phi(\ell\ell,\not{E}_{T})$ and $\not{E}_{T}$ to further distinguish the signal, and for $m_{X}=80\leavevmode\nobreak\ \mathrm{GeV}$, we found $M_{T2}$ to be a more useful variable. Each cut has been optimized such that it gives the highest significance, defined in Eq. (15).

Appendix B Reproducing the CKM Matrix and Flavor Changing Interactions of $X$

This model was first suggested in Ref. 1705.01822 , and the details of the model are somewhat complicated and lengthy. Rather than discussing all of the moving parts, we will focus on the generation of the CKM matrix and potential FCNC.

Because the third generation is charged under $U(1)_{B-L}^{(3)}$ while the first and second generations are not, mixing among generations requires $\phi$. The full Yukawa interaction, including interactions with Higgs or $\phi$ and working in a basis with diagonal kinetic terms, can be written as:

[TABLE]

The upper $2\times 2$ block of both quark mass matrices can be brought to diagonal form by rotations among $Q_{1,2},u_{R,1,2}$ and $d_{R,1,2}$. Note that, after these rotations – call them $R_{12}^{uL,dL}$, the up-type quark mass matrix has non-zero $(1,3),(2,3)$ entries, while the down-type matrix has the opposite structure:

[TABLE]

This structure follows automatically from the $U(1)_{B-L}^{(3)}$ charge of $\phi$. Given this structure, bringing the mass matrices to fully diagonal form can be accomplished by redefinitions among left handed fermions between $Q_{3}$ and $Q_{1,2}$ and redefinitions among the right handed down quarks between $d_{R3}$ and $d_{R,1,2}$. As the kinetic terms of the three generations are not identical, these last redefinitions generically induce FCNC in $U(1)_{B-L}^{(3)}$ gauge interactions. These FCNC are tightly constrained, especially in the down-quark sector. However, if we impose that $a,\,b\simeq 0$ in Eq. (21), all FCNC are relegated to the up-quark sector, where constraints are weaker. In this circumstance, a viable ($3\times 3$) CKM matrix is still generated, and one can show that the elements of the up-quark matrix in Eq. (21) are proportional to the CKM elements $c\,\sim V_{ub}$ and $d\,\sim V_{cb}$ 1705.01822 .

We emphasize that the choice $a\sim b\sim 0$ is not demanded by the setup, but is a phenomenological constraint. Accepting this constraint, we can work out the fermion mass basis interactions with $X$. The only place where FCNC $U(1)_{B-L}^{(3)}$ interactions occur is with left handed up-quarks. Specifically, expanding out the kinetic term and performing the rotations described above to go to the mass basis:

[TABLE]

There are no off-diagonal terms present in the $u_{R}$, $d_{R}$, or leptonic interactions with $X$, so they all have the same for as the $d_{L}$ interaction in Eq. (22).