New Signatures of Flavor Violating Higgs Couplings

TL;DR

This paper investigates novel LHC signatures from flavor violating Higgs couplings involving top quarks and tau leptons, proposing new search strategies and constraining these couplings using existing data.

Contribution

It introduces new search strategies for flavor violating Higgs processes and provides constraints on such couplings from current LHC data.

Findings

Detection of $pp \to thh$ possible at high luminosity LHC

Constraints on $pp \to tH^0$ with various decay channels

Constraints on $H^0 \to \tau\mu$ from CMS excess

Abstract

We explore several novel LHC signatures arising from quark or lepton flavor violating couplings in the Higgs sector, and we constrain such couplings using LHC data. Since the largest signals are possible in channels involving top quarks or tau leptons, we consider in particular the following flavor violating processes: (1) $pp \to thh$ (top plus di-Higgs final state) arising from a dimension six coupling of up-type quarks to three insertions of the Higgs field. We develop a search strategy for this final state and demonstrate that detection is possible at the high luminosity LHC if flavor violating top--up--Higgs couplings are not too far below the current limit. (2) $pp \to tH^0$, where $H^0$ is the heavy neutral CP-even Higgs boson in a two Higgs doublet model (2HDM). We consider the decay channels $H^0 \to tu, WW, ZZ, hh$ and use existing LHC data to constrain the first three of them. For the fourth, we adapt our search for the $thh$ final state, and we demonstrate that in large regions of the parameter space, it is superior to other searches, including searches for flavor violating top quark decays ($t \to hq$). (3) $H^0 \to \tau\mu$, again in the context of a 2HDM. This channel is particularly well motivated by the recent CMS excess in $h \to \tau\mu$, and we use the data from this search to constrain the properties of $H^0$.