Observation of $\psi(3686) \to p \bar{p} \eta^{\prime}$ and improved measurement of $J/\psi \to p \bar{p} \eta^{\prime}$

TL;DR

This paper reports the first observation of the decay process $(3686)  p ar{p} ta^{}$ with high significance and provides improved measurements of the branching fractions for $J//(3686)$ decays, along with the $ta-ta^{}$ mixing angle.

Contribution

First observation of $(3686)  p ar{p} ta^{}$ decay and improved measurement of $J/$ decay branching fractions, including determination of the $ta-ta^{}$ mixing angle.

Findings

Branching fraction $B((3686)  p ar{p} ta^{}) = (1.10 \u00b1 0.10  0.08) imes 10^{-5}$

Branching fraction $B(J/  p ar{p} ta^{}) = (1.26 \u00b1 0.02  0.07) imes 10^{-4}$

Determined $ta-ta^{}$ mixing angles of approximately $-24^\u00b0$ with uncertainties.

Abstract

We observe the process $\psi(3686) \to p \bar{p} \eta^{\prime}$ for the first time, with a statistical significance higher than 10$\sigma$, and measure the branching fraction of $J/\psi \to p \bar{p} \eta^{\prime}$ with an improved accuracy compared to earlier studies. The measurements are based on $4.48 \times 10^8$ $\psi(3686)$ and $1.31 \times 10^{9}$ $J/\psi$ events collected by the BESIII detector operating at the BEPCII. The branching fractions are determined to be $B(\psi(3686) \to p \bar{p} \eta^{\prime}) = (1.10\pm0.10\pm0.08)\times10^{-5}$ and $B(J/\psi \to p \bar{p} \eta^{\prime})=(1.26\pm0.02\pm 0.07)\times10^{-4}$, where the first uncertainties are statistical and the second ones systematic. Additionally, the $\eta-\eta^{\prime}$ mixing angle is determined to be $-24^{\circ} \pm 11^{\circ}$ based on $\psi(3686) \to p \bar{p} \eta^{\prime}$, and $-24^{\circ} \pm 9^{\circ}$ based on $J/\psi \to p \bar{p} \eta^{\prime}$, respectively.

Tables

Table 1. Table 1: Results from separate and simultaneous fits for each decay, N sig subscript 𝑁 sig N_{\rm sig} is the number of signal events, ϵ italic-ϵ \epsilon is the selection efficiency, and ℬ ℬ \mathcal{B} is the branching fraction of ψ ​ ( 3686 ) → p ​ p ¯ ​ η ′ → 𝜓 3686 𝑝 ¯ 𝑝 superscript 𝜂 ′ \psi(3686)\to p\bar{p}\eta^{\prime} or J / ψ → p ​ p ¯ ​ η ′ → 𝐽 𝜓 𝑝 ¯ 𝑝 superscript 𝜂 ′ J/\psi\to p\bar{p}\eta^{\prime} .

	$\psi \left(3686\right)\to p\overline{p}{\eta }^{\prime }$			$J/\psi \to p\overline{p}{\eta }^{\prime }$
Category	$N_{\mathrm{sig}}$	$ϵ$ (%)	$ℬ$ (${10}^{-5}$)	$N_{\mathrm{sig}}$	$ϵ$ (%)	$ℬ$ (${10}^{-4}$)
${\eta }^{\prime }\to \gamma {\pi }^{+}{\pi }^{-}$	$337\pm 29$	22.6	$1.12\pm 0.10\pm 0.10$	$12390\pm 138$	25.5	$1.27\pm 0.02\pm 0.08$
${\eta }^{\prime }\to \eta {\pi }^{+}{\pi }^{-}$	$154\pm 14$	18.7	$1.07\pm 0.10\pm 0.08$	$3931\pm 74$	14.3	$1.24\pm 0.03\pm 0.10$
Simultaneous fit	–	–	$1.10\pm 0.10\pm 0.08$	–	–	$1.26\pm 0.02\pm 0.07$

Table 2. Table 2: Summary of relative systematic uncertainties (in %) in the branching fractions. The total systematic uncertainty is obtained by summing all the contributions from each source taking the correlations into account. Here I (II) is ψ ​ ( 3686 ) 𝜓 3686 \psi(3686) decay channel with η ′ → γ ​ π + ​ π − → superscript 𝜂 ′ 𝛾 superscript 𝜋 superscript 𝜋 \eta^{\prime}\to\gamma\pi^{+}\pi^{-} ( η ′ → η ​ π + ​ π − → superscript 𝜂 ′ 𝜂 superscript 𝜋 superscript 𝜋 \eta^{\prime}\to\eta\pi^{+}\pi^{-} ), and III (IV) is J / ψ 𝐽 𝜓 J/\psi decay channel with η ′ → γ ​ π + ​ π − → superscript 𝜂 ′ 𝛾 superscript 𝜋 superscript 𝜋 \eta^{\prime}\to\gamma\pi^{+}\pi^{-} ( η ′ → η ​ π + ​ π − → superscript 𝜂 ′ 𝜂 superscript 𝜋 superscript 𝜋 \eta^{\prime}\to\eta\pi^{+}\pi^{-} ).

Source	I	II	III	IV
Tracking	2.7	2.7	2.8	2.8
Photon	1.0	2.0	1.0	2.0
$\eta$ reconstruction	–	1.0	–	1.0
PID	2.3	2.3	2.5	2.5
Kinematic fit	1.9	2.4	1.7	2.3
Mass window	4.9	1.8	–	–
Branching fraction	1.7	1.7	1.7	1.7
Number of $𝝍$ events	0.7		0.6
Fit range	1.4		0.6
Signal shape	3.9		1.2
Background shape	2.7		1.3
Physics model	1.5		3.4
Sum	7.2		5.4

Table 3. Table 3: The various ratios between ψ ​ ( 3686 ) 𝜓 3686 {\boldmath{\psi(3686)}} and J / ψ 𝐽 𝜓 {\boldmath{J/\psi}} decays. ℬ p ​ p ¯ ​ η ′ subscript ℬ 𝑝 ¯ 𝑝 superscript 𝜂 ′ \mathcal{B}_{p\bar{p}\eta^{\prime}} and ℬ p ​ p ¯ ​ η subscript ℬ 𝑝 ¯ 𝑝 𝜂 \mathcal{B}_{p\bar{p}\eta} are the branching fractions, Ω p ​ p ¯ ​ η ′ subscript Ω 𝑝 ¯ 𝑝 superscript 𝜂 ′ \Omega_{p\bar{p}\eta^{\prime}} and Ω p ​ p ¯ ​ η subscript Ω 𝑝 ¯ 𝑝 𝜂 \Omega_{p\bar{p}\eta} are the three-body phase space factors  pdg_2014 , ℳ p ​ p ¯ ​ η ′ subscript ℳ 𝑝 ¯ 𝑝 superscript 𝜂 ′ \mathcal{M}_{p\bar{p}\eta^{\prime}} and ℳ p ​ p ¯ ​ η subscript ℳ 𝑝 ¯ 𝑝 𝜂 \mathcal{M}_{p\bar{p}\eta} are the accordingly determined matrix elements. The | ℳ p ​ p ¯ ​ η ′ / ℳ p ​ p ¯ ​ η | subscript ℳ 𝑝 ¯ 𝑝 superscript 𝜂 ′ subscript ℳ 𝑝 ¯ 𝑝 𝜂 |\mathcal{M}_{p\bar{p}\eta^{\prime}}/\mathcal{M}_{p\bar{p}\eta}| is calculated by ℬ p ​ p ¯ ​ η ′ ℬ p ​ p ¯ ​ η ⋅ Ω p ​ p ¯ ​ η Ω p ​ p ¯ ​ η ′ ⋅ subscript ℬ 𝑝 ¯ 𝑝 superscript 𝜂 ′ subscript ℬ 𝑝 ¯ 𝑝 𝜂 subscript Ω 𝑝 ¯ 𝑝 𝜂 subscript Ω 𝑝 ¯ 𝑝 superscript 𝜂 ′ \sqrt{\frac{\mathcal{B}_{p\bar{p}\eta^{\prime}}}{\mathcal{B}_{p\bar{p}\eta}}\cdot\frac{\Omega_{p\bar{p}\eta}}{\Omega_{p\bar{p}\eta^{\prime}}}} .

Ratio	$\psi \left(3686\right)$	$J/\psi$
${ℬ}_{p\overline{p}{\eta }^{\prime }}/{ℬ}_{p\overline{p}\eta }$ (%)	$18.3\pm 2.5$	$6.3\pm 0.6$
${\Omega }_{p\overline{p}{\eta }^{\prime }}/{\Omega }_{p\overline{p}\eta }$	0.5	0.2
$\left|{ℳ}_{p\overline{p}{\eta }^{\prime }}/{ℳ}_{p\overline{p}\eta }\right|$	$0.61\pm 0.04$	$0.56\pm 0.03$

Equations

$$\mathcal{B}(\psi (3686)\to p\bar{p}{\eta }^{\prime })$$

$$\mathcal{B}(J/\psi \to p\bar{p}{\eta }^{\prime })$$

Full text

I INTRODUCTION

Quantum Chromodynamics (QCD), the theory describing the strong interaction, has been tested thoroughly at high energy. However, in the medium energy region, theoretical calculations based on first principles are still unreliable since the non-perturbative contribution is significant and calculations have to rely on models. Experimental measurements in this energy region are helpful to validate or falsify models, constrain parameters, and inspire new calculations. Charmonium states are on the boundary between perturbative and non-perturbative regimes in QCD, therefore, their decays, especially the hadronic decays, provide ideal inputs to study the QCD. The availability of very large samples of vector charmonia, produced via electron-positron annihilation, such as $J/\psi$ and $\psi(3686)$, makes experimental studies of rare processes and decay channels with complicated intermediate structures possible.

Among these hadronic decays, scenarios of $\psi$ (in the following, $\bm{\psi}$ denotes either $J/\psi$ or $\psi(3686)$) decaying into baryon pairs have been understood via $c\bar{c}$ annihilation into three gluons or a virtual photon psi_to_BBbar . But its natural extension, the three-body decays, $\bm{\psi}\to p\bar{p}P$, where $P$ represents a pseudoscalar meson such as $\pi^{0}$, $\eta$, or $\eta^{\prime}$, still need more studies since intermediate states contribute significantly here. Specific models based on nucleon and $N^{*}$ pole diagrams have been proposed to deal with these problems soft_pion_theorem ; nucleon_pole ; nucleon_resonance . However, recent studies have focused on the final states $p\bar{p}\pi^{0}$ and $p\bar{p}\eta$, and not so much on $p\bar{p}\eta^{\prime}$, partially due to the limited experimental measurements.

The study of the process $\bm{\psi}\to p\bar{p}\eta^{\prime}$, as well as the branching fraction of $\bm{\psi}\to p\bar{p}\eta$, can also be used to determine the $\eta-\eta^{\prime}$ mixing angle $\theta_{\eta-\eta^{\prime}}$. The $\eta-\eta^{\prime}$ mixing angle, which was proposed in quark model SU(3) flavor symmetry soft_pion_theorem , is expected to be $-(10^{\circ}\sim 17^{\circ})$ based on a QCD inspired calculation soft_pion_theorem or $-(13^{\circ}\sim 16^{\circ})\pm 6^{\circ}$ based on the quark-line rule (QLR) quark-line-rule .

In addition, using the process $\bm{\psi}\to p\bar{p}\eta^{\prime}$, we are able to test the “12% rule”. The ratio $Q$ of the branching fractions of $J/\psi$ and $\psi(3686)$ can be written in terms of their total and leptonic widths under the assumption that the charmonium systems are non-relativistic and decay to hadrons predominantly via point-like annihilation into three gluons 12_rule_01 ; Augustin:1974xw : $Q=\frac{\mathcal{B}(\psi(3686)\to ggg)}{\mathcal{B}(J/\psi\to ggg)}=\frac{\Gamma(\psi(3686)\to e^{+}e^{-})\cdot\Gamma(J/\psi)}{\Gamma(J/\psi\to e^{+}e^{-})\cdot\Gamma(\psi(3686))}=(12.2\pm 2.4)\%$ rhopi_puzzle_mrkii . This relation was extended to exclusive processes later, ignoring other factors associated with each exclusive mode such as multiplicity and phase space factors. Although the “12% rule” has been confirmed experimentally for many decay modes, severe violation has been found in several channels rhopi_puzzle_mrkii . Many theoretical explanations rho_pi_rev have been proposed to explain the violation of the “12% rule”, but none is satisfactory.

The baryonic three-body decay $J/\psi\to p\bar{p}\eta^{\prime}$ was first observed by MARKI with branching fraction $(1.8\pm 0.6)\times 10^{-3}$ in 1978 jpsi_ppbetap_mrki and confirmed by MARKII with branching fraction $(0.68\pm 0.23\pm 0.17)\times 10^{-3}$ in 1984 jpsi_ppbetap_mrkii . Later, using $5.80\times 10^{7}$ $J/\psi$ events, BESII performed a further measurement of the branching fraction of $J/\psi\to p\bar{p}\eta^{\prime}$ with $(2.00\pm 0.23\pm 0.28)\times 10^{-4}$ jpsi_ppbetap_besii . The process $\psi(3686)\to p\bar{p}\eta^{\prime}$ has not been observed so far.

At present, two large samples of $4.48\times 10^{8}$ $\psi(3686)$ events psip_num and $1.31\times 10^{9}$ $J/\psi$ events jpsi_num have been collected with the BESIII detector. Using these large data sets, we present the first observation of $\psi(3686)\to p\bar{p}\eta^{\prime}$ and an improved measurement of the branching fraction of $J/\psi\to p\bar{p}\eta^{\prime}$. In this analysis, the $\eta^{\prime}$ candidates are reconstructed via their decays $\eta^{\prime}\to\gamma\pi^{+}\pi^{-}$ and $\eta^{\prime}\to\pi^{+}\pi^{-}\eta,\eta\to\gamma\gamma$.

II BESIII DETECTOR AND DATA SAMPLES

The BESIII detector, described in detail in Ref. detail_bepcii , has a geometrical acceptance of 93% of 4$\pi$ solid angle. It can be subdivided into four main sub-detectors. A helium-based multi-layer drift chamber (MDC) determines the momentum of charged particles, traveling through a 1 T (for $J/\psi$ sample 0.9 T in 2012) magnetic field, with a resolution 0.5% at 1 GeV/$c$, as well as the ionization energy loss (d$E$/d$x$) with a resolution better than 6.0% for electrons from Bhabha scattering. A time-of-flight system (TOF) made of plastic scintillators with a time resolution of 80 ps (110 ps) in the barrel (end caps) is used for particle identification (PID). An electromagnetic calorimeter (EMC) consisting of 6240 CsI(Tl) crystals measures the energies of photons with a resolution of 2.5% (5.0%) in the barrel (end caps) at 1 GeV, and their positions with a resolution of 6 mm (9 mm) in the barrel (end caps). A muon counter (MUC) based on resistive plate chambers with 2 cm position resolution provides information for muon identification.

A geant4-based geant4_01 ; geant4_02 detector simulation package is used to model the detector response. Inclusive Monte Carlo (MC) samples, including $5.06\times 10^{8}$ $\psi(3686)$ and $1.23\times 10^{9}$ $J/\psi$ events, are used for background studies. The production of the $\psi(3686)$ and $J/\psi$ resonances is simulated using the event generator kkmc kkmc_01 ; kkmc_02 , and their decays are simulated by evtgen evtgen for those with a known branching fraction obtained from the Particle Data Group (PDG) pdg_2014 and by the lundcharm model lundcharm for unmeasured ones. Signal MC samples are generated to determine the detection efficiency and to optimize selection criteria. The $\psi(3686)\to p\bar{p}\eta^{\prime}$, $J/\psi\to p\bar{p}\eta^{\prime}$, $\eta^{\prime}\to\eta\pi^{+}\pi^{-}$ and $\eta\to\gamma\gamma$ are generated according to phase space (PHSP) distributions, while $\eta^{\prime}\to\gamma\pi^{+}\pi^{-}$ according to the model-dependent amplitudes determined in Ref. diy_test3 . Data samples taken at center-of-mass energies $\sqrt{s}=3.080$ and $3.650$ GeV with integrated luminosities of $31$ pb*-1* and $44$ pb*-1*, respectively, are used to estimate continuum backgrounds.

III EVENT SELECTIONS AND BACKGROUND ANALYSIS

Charged tracks are reconstructed from hits in the MDC. For each track, the polar angle must satisfy $|\cos\theta|<0.93$ and the point of closest approach to the interaction point must be within $\pm 1$ cm in the plane perpendicular to the beam and $\pm 10$ cm along the beam direction. The TOF and d$E$/d$x$ information is combined to calculate PID likelihoods for the pion and proton hypotheses, and the PID hypothesis with the largest likelihood is assigned to the track.

Photons are reconstructed from isolated electromagnetic showers in the EMC. The angle between the directions of any cluster and its nearest charged track must be larger than 10 or 30 degrees to pion or (anti-)proton tracks, respectively. The efficiency and energy resolution are improved by including the energy deposited in nearby TOF counters. A photon candidate must deposit at least 25 MeV (50 MeV) in the barrel (end caps) region, corresponding to an angular coverage of $|\cos\theta|<0.80$ ($0.86<|\cos\theta|<0.92$). The timing information obtained from the EMC is required to be $0\leq t_{\text{EMC}}\leq 700$ ns to suppress electronic noise and beam backgrounds unrelated to the event.

Signal candidates must have four charged tracks identified as $p$, $\bar{p}$, $\pi^{+}$ and $\pi^{-}$, as well as at least one (two) photon(s) for the $\eta^{\prime}\to\gamma\pi^{+}\pi^{-}\ (\eta^{\prime}\to\eta\pi^{+}\pi^{-})$ mode. The events with $920<M_{\gamma\pi^{+}\pi^{-}(\eta\pi^{+}\pi^{-})}<1000$ MeV/$c^{2}$ are accepted for further analysis, where $M_{\gamma\pi^{+}\pi^{-}(\eta\pi^{+}\pi^{-})}$ is the invariant mass of $\gamma\pi^{+}\pi^{-}$ or $\eta\pi^{+}\pi^{-}$, respectively. To improve the resolution and to suppress backgrounds, a kinematic fit to all final state particles with a constraint on the initial $e^{+}e^{-}$ four-momentum is performed. In addition, for the $\eta^{\prime}\to\pi^{+}\pi^{-}\eta$ mode, the invariant mass of the two photons is constrained to the nominal mass of $\eta$. The $\chi^{2}$ of the kinematic fit for each decay mode is required to be less than an optimized value obtained by maximizing the figure of merit $S/\sqrt{S+B}$, where $S$ is the number of signal events from a signal MC sample normalized to the preliminary measurements and $B$ is the number of background events in the $\eta^{\prime}$ signal region obtained from inclusive MC samples. When there are more photon candidates than required in an event, we loop over all possible combinations and keep the one with the smallest kinematic fit $\chi^{2}$. After the kinematic fit, the $\eta^{\prime}$ signal region is defined as $948.2<M_{\gamma\pi^{+}\pi^{-}(\eta\pi^{+}\pi^{-})}<967.4$ MeV/$c^{2}$, while the sideband regions are defined as $932.4<M_{\gamma\pi^{+}\pi^{-}(\eta\pi^{+}\pi^{-})}<942.0$ MeV/$c^{2}$ and $974.0<M_{\gamma\pi^{+}\pi^{-}(\eta\pi^{+}\pi^{-})}<983.6$ MeV/$c^{2}$.

To remove background events, we apply the following requirements:

• •

For $\psi(3686)\to p\bar{p}\eta^{\prime}$ with $\eta^{\prime}\to\gamma\pi^{+}\pi^{-}$, we veto $\psi(3686)\to\gamma\chi_{cJ},\chi_{cJ}\to p\bar{p}\pi^{+}\pi^{-}$ ($J$ = 0, 1, 2) decays by requiring $|M_{\gamma}^{\rm rec}-m_{\chi_{c0}}|>40$ MeV/$c^{2}$, $|M_{\gamma}^{\rm rec}-m_{\chi_{c1}}|>14$ MeV/$c^{2}$ and $|M_{\gamma}^{\rm rec}-m_{\chi_{c2}}|>15$ MeV/$c^{2}$, and veto $\psi(3686)\to\pi^{+}\pi^{-}J/\psi,J/\psi\to\gamma p\bar{p}$ decays by requiring $|M^{{\rm{rec}}}_{\pi^{+}\pi^{-}}-m_{J/\psi}|>8$ MeV/$c^{2}$.

• •

For $\psi(3686)\to p\bar{p}\eta^{\prime}$ with $\eta^{\prime}\to\eta\pi^{+}\pi^{-}$, we veto $\psi(3686)\to\eta J/\psi,\eta\to\gamma\gamma,J/\psi\to p\bar{p}\pi^{+}\pi^{-}$ decays by requiring $|M^{{\rm{rec}}}_{\gamma\gamma}-m_{J/\psi}|>8$ MeV/$c^{2}$, and veto $\psi(3686)\to\pi^{+}\pi^{-}J/\psi,J/\psi\to\eta p\bar{p}$ decays by requiring $|M^{{\rm{rec}}}_{\pi^{+}\pi^{-}}-m_{J/\psi}|>8$ MeV/$c^{2}$.

Here, $M_{\gamma}^{\rm rec}$, $M^{{\rm{rec}}}_{\pi^{+}\pi^{-}}$, and $M^{{\rm{rec}}}_{\gamma\gamma}$ are the recoil mass of $\gamma$, $\pi^{+}\pi^{-}$, and $\gamma\gamma$, while $m_{\chi_{cJ}}$ and $m_{J/\psi}$ are the nominal $\chi_{cJ}$ and $J/\psi$ masses pdg_2014 , respectively. The mass window for each requirement is determined based on the exclusive MC simulation.

The backgrounds from $\psi(3686)$ and $J/\psi$ decays are studied with inclusive MC samples. For $\psi(3686)\to p\bar{p}\eta^{\prime}$ with $\eta^{\prime}\to\gamma\pi^{+}\pi^{-}$, even after the $\chi_{cJ}$ mass window requirements, the main remaining background is the decay $\psi(3686)\to\gamma\chi_{cJ}$ with $\chi_{cJ}\to p\bar{p}\pi^{+}\pi^{-}$, which has the same final state as the signal process. A study of MC simulated events shows that the $M_{\gamma\pi^{+}\pi^{-}}$ distribution from $\psi(3686)\to\gamma\chi_{cJ}\to\gamma p\bar{p}\pi^{+}\pi^{-}$ is smooth. Therefore, its contribution can be easily determined in a fit. For the other three decay modes, $\psi(3686)\to p\bar{p}\eta^{\prime}$ with $\eta^{\prime}\to\eta\pi^{+}\pi^{-}$, $J/\psi\to p\bar{p}\eta^{\prime}$ with $\eta^{\prime}\to\gamma\pi^{+}\pi^{-}$ and $\eta^{\prime}\to\eta\pi^{+}\pi^{-}$, there are no dominant background processes, but many decay channels with a small contribution each. The backgrounds from the continuum process $e^{+}e^{-}\to q\bar{q}$ are studied with data samples taken at $\sqrt{s}=3.080$ and $3.650$ GeV. The background level is found to be very low, and the background events do not peak in the signal region.

The $M_{\gamma\pi^{+}\pi^{-}}$ and $M_{\eta\pi^{+}\pi^{-}}$ distributions of the events that pass all selection criteria are shown in Fig. 1. Peaks originating from $\eta^{\prime}$ decays are observed. Figure 2 shows the Dalitz plots of the events in the $\eta^{\prime}$ signal region, and Figs. 3 and 4 show the invariant mass projections, where the side band backgrounds have been subtracted. Based on these plots, no obvious intermediate structures in invariant mass of $p\eta^{\prime}$, $\bar{p}\eta^{\prime}$, or $p\bar{p}$ are observed.

IV SIGNAL YIELDS AND BRANCHING FRACTIONS

To determine the branching fractions, simultaneous unbinned maximum likelihood fits to the $\gamma\pi^{+}\pi^{-}$ and $\eta\pi^{+}\pi^{-}$ invariant mass spectra are performed for the $\psi(3686)$ data and for the $J/\psi$ data. The signal shape is represented by the MC-simulated $\eta^{\prime}$ mass distribution, convolved with a Gaussian function with free mean and width to account for the mass and resolution difference between data and MC simulation. The background is parameterized as a second-order Chebyshev polynomial with free parameters. In the simultaneous fit, the ratio of the number of $\eta^{\prime}\to\gamma\pi^{+}\pi^{-}$ events to that of $\eta^{\prime}\to\eta\pi^{+}\pi^{-}$ events is fixed to $\frac{\mathcal{B}(\eta^{\prime}\to\gamma\pi^{+}\pi^{-})\cdot\epsilon_{\eta^{\prime}\to\gamma\pi^{+}\pi^{-}}}{\mathcal{B}(\eta^{\prime}\to\pi^{+}\pi^{-}\eta)\cdot\mathcal{B}(\eta\to\gamma\gamma)\cdot\epsilon_{\eta^{\prime}\to\pi^{+}\pi^{-}\eta}}$, where $\epsilon_{\eta^{\prime}\to\gamma\pi^{+}\pi^{-}}$ and $\epsilon_{\eta^{\prime}\to\pi^{+}\pi^{-}\eta}$ are the global efficiencies for each $\eta^{\prime}$ decay mode. Due to differences in tracking and PID efficiencies between data and MC simulation for protons and anti-protons, the MC-determined global efficiencies are corrected by multiplying factors 1.030 (1.038) and 0.980 (0.984) for tracking and PID, respectively, for $\psi(3686)\to p\bar{p}\eta^{\prime}$ ($J/\psi\to p\bar{p}\eta^{\prime}$). These correction factors are ratios of efficiencies between data and MC simulation obtained by studying the control samples $\bm{\psi}\to p\bar{p}\pi^{+}\pi^{-}$, where the efficiencies are weighted according to the distributions of transverse momentum (for tracking) or momentum (for PID) of protons and anti-protons.

The results of the fits are listed in Table 1 and shown in Fig. 1. The goodness of the fit is $\chi^{2}/{\rm ndf}=51.50/44=1.17$ for $\psi(3686)\to p\bar{p}\eta^{\prime}$ with $\eta^{\prime}\to\gamma\pi^{+}\pi^{-}$, $37.85/44=0.86$ for $\psi(3686)\to p\bar{p}\eta^{\prime}$ with $\eta^{\prime}\to\eta\pi^{+}\pi^{-}$, $234.75/194=1.21$ for $J/\psi\to p\bar{p}\eta^{\prime}$ with $\eta^{\prime}\to\gamma\pi^{+}\pi^{-}$, and $205.66/194=1.06$ for $J/\psi\to p\bar{p}\eta^{\prime}$ with $\eta^{\prime}\to\eta\pi^{+}\pi^{-}$. The resolution difference between data and MC simulation is 1 MeV in each decay mode. The branching fractions are determined to be

[TABLE]

Here the first uncertainties are statistical and the second ones systematic, as discussed in Sec. V. As a cross check, we also fit these channels separately, and Table 1 lists the signal yields, the selection efficiencies, and the branching fractions obtained for each decay mode. The branching fractions obtained from the simultaneous and separate fits are consistent with each other.

V SYSTEMATIC UNCERTAINTIES

The systematic uncertainties mainly come from the MDC tracking, photon and $\eta$ reconstruction, PID, kinematic fit, mass windows, branching fractions of the decay modes used to reconstruct the $\eta^{\prime}$, the number of $\psi$ decays, fitting procedure, and the physics model used to determine the efficiency. All the contributions are given in Table 2. The overall systematic uncertainties are obtained by adding all systematic uncertainties, taking the correlations into account.

The uncertainty in the MDC tracking efficiency for each pion is estimated with the control sample $\psi(3686)\to\pi^{+}\pi^{-}J/\psi$, and a 1.0% systematic uncertainty per pion is obtained tracking_pion . This gives a total of 2.0% for each decay mode. The tracking efficiencies of protons and anti-protons are studied with the control sample $\bm{\psi}\to p\bar{p}\pi^{+}\pi^{-}$. The MC efficiencies for the signal processes are corrected using the results from the control samples, and the uncertainties of the tracking differences between data and MC simulation are taken as the systematic uncertainties, which are 0.7% (1.0%) per proton (anti-proton) for the $\psi(3686)$ decays, and 0.9% (1.0%) per proton (anti-proton) for the $J/\psi$ decays. Assuming the uncertainties of proton and anti-proton are totally correlated, the tracking uncertainty of a proton–anti-proton pair is 1.7% (1.9%) for the $\psi(3686)$ ($J/\psi$) data sample. The tracking efficiencies of pion and proton are independent, and the total tracking uncertainty is determined to be 2.7% (2.8%) for the $\psi(3686)$ ($J/\psi$) samples.

The uncertainty in the photon reconstruction is studied by using the control sample $J/\psi\to\rho^{0}\pi^{0}$, and a 1.0% systematic uncertainty is estimated for each photon photon_rec . The uncertainty of the $\eta$ reconstruction from $\gamma\gamma$ final state is 1.0% per $\eta$, as determined from a high purity control sample of $J/\psi\to p\bar{p}\eta$ eta-rec .

The uncertainty in the PID efficiency for pions is estimated to be 1.0% per pion pid_pion , and the total PID uncertainty for two pions is 2.0% for each decay mode. The efficiencies of the proton and the anti-proton identification are studied with the control samples $\bm{\psi}\to p\bar{p}\pi^{+}\pi^{-}$. The MC efficiencies for the signal processes are corrected using the results from the control samples, and the uncertainties of the corrections are taken as the systematic uncertainties, which are 0.5% (0.6%) per proton (anti-proton) for the $\psi(3686)$, and 0.6% (0.8%) per proton (anti-proton) for the $J/\psi$ samples. Assuming the uncertainties of proton and anti-proton are totally correlated, the PID uncertainty of a proton and anti-proton pair is calculated to be 1.1%(1.4%) for the $\psi(3686)$ ($J/\psi$) data samples. The PID efficiencies of pion and proton are independent, and the total PID uncertainty is determined to be 2.3% (2.5%) for the $\psi(3686)$ ($J/\psi$) samples.

The uncertainty associated with the kinematic fit is estimated by comparing the efficiencies with or without a helix parameter correction applied to simulated data bib23 . Control samples $J/\psi\to pK^{-}\bar{\Lambda}+c.c.$ and $\psi(3686)\rightarrow K^{+}K^{-}\pi^{+}\pi^{-}$ bib24 are used to obtain the correction to the track helix parameters. The uncertainties due to the kinematic fit are determined to be 1.9%, 2.4%, 1.7%, and 2.3%, for $\psi(3686)\to p\bar{p}\eta^{\prime}$ with $\eta^{\prime}\to\gamma\pi^{+}\pi^{-}$, and $\psi(3686)\to p\bar{p}\eta^{\prime}$ with $\eta^{\prime}\to\eta\pi^{+}\pi^{-}$, $J/\psi\to p\bar{p}\eta^{\prime}$ with $\eta^{\prime}\to\gamma\pi^{+}\pi^{-}$, and $J/\psi\to p\bar{p}\eta^{\prime}$ with $\eta^{\prime}\to\eta\pi^{+}\pi^{-}$, respectively.

The uncertainty due to the mass windows used to veto background events originates from the differences in the mass resolutions between data and MC simulation. We repeat the analysis by enlarging or reducing the mass window. The largest difference is used as an estimate of the corresponding systematic uncertainty. The $\eta^{\prime}$ signal region and sideband regions are not used to veto the background events, so they have no effect on the branching fraction determination. The uncertainties due to different mass windows are considered to be independent, so we add them in quadrature. For the decay $\psi(3686)\to p\bar{p}\eta^{\prime}$ with $\eta^{\prime}\to\gamma\pi^{+}\pi^{-}$ ($\eta^{\prime}\to\eta\pi^{+}\pi^{-}$), the uncertainty is 4.9% (1.8%). The uncertainty for the mass-window selection for $J/\psi\to p\bar{p}\eta^{\prime}$ is found negligible.

The systematic uncertainties due to the branching fractions of the subsequent $\eta^{\prime}$ and $\eta$ decays are 1.7% for decays $\eta^{\prime}\to\gamma\pi^{+}\pi^{-}$ and $\eta^{\prime}\to\eta\pi^{+}\pi^{-}\to\gamma\gamma\pi^{+}\pi^{-}$ pdg_2014 . The numbers of $\psi(3686)$ and $J/\psi$ events have been estimated via inclusive hadronic events with relative uncertainties of 0.7% psip_num and 0.6% jpsi_num , respectively.

The fit range, signal shape, and background shape are considered as the sources of the systematic uncertainty related with the fit procedure. In the nominal fit, the mass range is $[0.90,1.04]$ GeV/$c^{2}$, and we repeat the fit by changing the range by $\pm$10 MeV/$c^{2}$. The largest change in the final result is taken as the uncertainty due to the fit range, which is 1.4% and 0.6% for $\psi(3686)\to p\bar{p}\eta^{\prime}$ and $J/\psi\to p\bar{p}\eta^{\prime}$, respectively. For the signal shape, we change the nominal shape to a double-Gaussian or a Breit-Wigner function convolved with a Gaussian function, and the largest difference from the nominal result is taken as the uncertainty of the signal shape, which is 3.9% and 1.2% for $\psi(3686)\to p\bar{p}\eta^{\prime}$ and $J/\psi\to p\bar{p}\eta^{\prime}$, respectively. We replace the background shape with a first-order Chebyshev or second-order polynomial function, the largest differences from the nominal result, 2.7% and 1.3% for $\psi(3686)\to p\bar{p}\eta^{\prime}$ and $J/\psi\to p\bar{p}\eta^{\prime}$, respectively, are taken as systematic uncertainties.

The signal MC sample is generated assuming pure phase space distribution, in which possible intermediate states and non-flat angular distributions are ignored. Although no strong structure is visible in the Dalitz plots shown in Fig. 2, the phase space MC does not provide a very good description of the data, as shown in Fig. 3 and Fig. 4, which results in a large systematic uncertainty, especially for the $J/\psi$ decay modes. For $\psi(3686)\to p\bar{p}\eta^{\prime}$, since the statistics are limited, we weight each MC-generated phase space event by the $M_{p\bar{p}}$ distribution, and a difference of 1.5% in the efficiency between the nominal and weighted MC samples is taken as the uncertainty. For $J/\psi\to p\bar{p}\eta^{\prime}$, we re-generate signal MC events based on body3 Ablikim:2014dqd , a data-driven MC generator, and a difference of 3.4% in the efficiencies is taken as the uncertainty.

VI SUMMARY AND DISCUSSION

Based on $4.48\times 10^{8}$ $\psi(3686)$ decays, we observe for the first time $\psi(3686)\to p\bar{p}\eta^{\prime}$, and measure its branching fraction to be $(1.10\pm 0.10\pm 0.08)\times 10^{-5}$. Based on $1.31\times 10^{9}$ $J/\psi$ decays, we obtain the most accurate measurement so far of $\mathcal{B}(J/\psi\to p\bar{p}\eta^{\prime})=(1.26\pm 0.02\pm 0.07)\times 10^{-4}$.

With the measurement of $J/\psi\to p\bar{p}$ pdg_2014 , we determine the ratio of decay widths $\frac{\Gamma(J/\psi\to p\bar{p}\eta^{\prime})}{\Gamma(J/\psi\to p\bar{p})}=(5.94\pm 0.35)\%$. It is larger than the nucleon pole contribution by two orders of magnitude according to Ref. nucleon_pole . This implies the validity of the $N^{*}$ pole hypothesis. However, from the invariant mass distributions of $p\eta^{\prime}$ and $\bar{p}\eta^{\prime}$ in Fig. 4, no distinctive structure is observed, which indicates that very broad intermediate $N^{*}$ states or other decay mechanisms are needed to explain the large ratio. Similarly, using the results in Ref. psip_to_NN , we determine the ratio $\frac{\Gamma(\psi(3686)\to p\bar{p}\eta^{\prime})}{\Gamma(\psi(3686)\to p\bar{p})}=(3.61\pm 0.41)\%$, where common uncertainties have been cancelled. This ratio will helpful for future studies of the nucleon and $N^{*}$ pole contributions in $\psi(3686)$ baryonic decays.

Combining our result with the branching fractions of $\bm{\psi}\to p\bar{p}\eta$ reported in Ref. pdg_2014 and following the procedure described in Ref. soft_pion_theorem , we determine an $\eta-\eta^{\prime}$ mixing angle of $-24^{\circ}\pm 11^{\circ}$ for the $\psi(3686)$ decays and $-24^{\circ}\pm 9^{\circ}$ for the $J/\psi$ decays. We observe that the two values are very similar, even though the uncertainties are in both cases very large. This might indicate a universal behavior of the $\eta-\eta^{\prime}$ mixing angle as expected. These results are consistent with the QCD-inspired calculations $\theta_{\eta-\eta^{\prime}}=-(17^{\circ}\sim 10^{\circ})$ soft_pion_theorem , and $-(16^{\circ}\sim 13^{\circ})\pm 6^{\circ}$ based on the quark-line rule quark-line-rule .

Table 3 shows the details for the ratios of branching fractions, three-body phase space factors, and determined matrix elements. The ratios of the matrix elements are consistent with the theoretical prediction that falls within the range $[0.5,~{}0.9]$ according to Ref. soft_pion_theorem .

Our results for the branching fractions of $\psi(3686)\to p\bar{p}\eta^{\prime}$ and $J/\psi\to p\bar{p}\eta^{\prime}$ result in the ratio $\frac{\mathcal{B}(\psi(3686)\to p\bar{p}\eta^{\prime})}{\mathcal{B}(J/\psi\to p\bar{p}\eta^{\prime})}=(8.7\pm 1.0)\%$, where the common uncertainties have been canceled. Even though the ratio is in reasonabe agreement with 12%, we note that the kinematics of the two processes are very different, and the “12% rule” may be too naive in this case. The phase space ratio is $\Omega_{\psi(3686)\to p\bar{p}\eta^{\prime}}/\Omega_{J/\psi\to p\bar{p}\eta^{\prime}}=8.13$, if any possible intermediate structure is ignored. Furthermore, the Dalitz plots of $J/\psi$ and $\psi(3686)$ decays, shown in Fig. 2, indicate that many events in $\psi(3686)$ decays, possibly via $N^{\ast}\bar{N}+c.c.$ intermediate states with $p\eta^{\prime}$ or $\bar{p}\eta^{\prime}$ mass greater than 2.13 GeV/$c^{2}$, are not kinematically possible in $J/\psi$ decays. Taken these factors into account, the $Q$ value is suppressed a lot, implying that the “12% rule” is violated significantly.

VII ACKNOWLEDGEMENTS

The BESIII collaboration thanks the staff of BEPCII and the IHEP computing center for their strong support. This work is supported in part by National Key Basic Research Program of China under Contract No. 2015CB856700; National Natural Science Foundation of China (NSFC) under Contracts Nos. 11335008, 11425524, 11625523, 11635010, 11735014; the Chinese Academy of Sciences (CAS) Large-Scale Scientific Facility Program; the CAS Center for Excellence in Particle Physics (CCEPP); Joint Large-Scale Scientific Facility Funds of the NSFC and CAS under Contracts Nos. U1532257, U1532258, U1732263; CAS Key Research Program of Frontier Sciences under Contracts Nos. QYZDJ-SSW-SLH003, QYZDJ-SSW-SLH040; 100 Talents Program of CAS; INPAC and Shanghai Key Laboratory for Particle Physics and Cosmology; German Research Foundation DFG under Contracts Nos. Collaborative Research Center CRC 1044, FOR 2359; Istituto Nazionale di Fisica Nucleare, Italy; Koninklijke Nederlandse Akademie van Wetenschappen (KNAW) under Contract No. 530-4CDP03; Ministry of Development of Turkey under Contract No. DPT2006K-120470; National Science and Technology fund; The Swedish Research Council; U. S. Department of Energy under Contracts Nos. DE-FG02-05ER41374, DE-SC-0010118, DE-SC-0010504, DE-SC-0012069; University of Groningen (RuG) and the Helmholtzzentrum fuer Schwerionenforschung GmbH (GSI), Darmstadt.