Search for a fourth generation b'-quark at LEP-II at sqrt{s}=196-209 GeV

TL;DR

This study searched for fourth-generation b'-quarks at LEP-II energies, setting upper limits on their production and decay probabilities, and constraining related CKM matrix elements, with no evidence of such particles found.

Contribution

First search for b'-quarks at LEP-II energies, providing upper limits on their production and decay, and constraining CKM matrix element ratios within a four-generation model.

Findings

No evidence for b'-quark production was observed.

Upper limits on branching ratios were established for b' decays.

Constraints on CKM matrix element ratios were derived.

Abstract

A search for the pair production of fourth generation b'-quarks was performed using data taken by the DELPHI detector at LEP-II. The analysed data were collected at centre-of-mass energies ranging from 196 to 209 GeV, corresponding to an integrated luminosity of 420 pb^{-1}. No evidence for a signal was found. Upper limits on BR(b' -> bZ) and BR(b' -> cW) were obtained for b' masses ranging from 96 to 103 GeV/c^2. These limits, together with the theoretical branching ratios predicted by a sequential four generations model, were used to constrain the value of R_{CKM}=|V_{cb'}/V_{tb'}V_{tb}|, where V_{cb'}, V_{tb'} and V_{tb} are elements of the extended CKM matrix.

Figures

Tables

Table 1. Table 1: The luminosity collected with the DELPHI detector at each centre-of-mass energy is shown. The energy bin labelled 206 ∗ corresponds to the data collected with one sector of the TPC turned off.

$\sqrt{s}$ (GeV)	196	200	202	205	207	206∗
luminosity (pb-1)	76.0	82.7	40.2	80.0	81.9	59.2

Table 2. Table 2: The final states considered in this analysis are shown. About 81% and 90% of the branching ratio to the FCNC and CC channels were covered, respectively.

$b^{\prime }$ decay	boson decays	$BR$ (%)	final states
$b^{\prime }\to bZ$ (FCNC)	$ZZ\to l^{+}{l}^{-}\nu \overline{\nu }$	4.0	$b\overline{b}{l}^{+}{l}^{-}\nu \overline{\nu }$
	$\mathrm{ZZ}\to q\overline{q}\nu \overline{\nu }$	28.0	$b\overline{b}q\overline{q}\nu \overline{\nu }$
	$\mathrm{ZZ}\to q\overline{q}q\overline{q}$	48.6	$b\overline{b}q\overline{q}q\overline{q}$
$b^{\prime }\to cW$ (CC)	$\mathrm{WW}\to q\overline{q}{l}^{+}\nu$	43.7	$c\overline{c}q\overline{q}{l}^{+}\nu$
	$\mathrm{WW}\to q\overline{q}q\overline{q}$	45.8	$c\overline{c}q\overline{q}q\overline{q}$

Table 3. Table 3: Summary of the final state assignment criteria.

final state	assignment criteria
$b\overline{b}{l}^{+}{l}^{-}\nu \overline{\nu }$	at least 1 isolated lepton
$b\overline{b}q\overline{q}\nu \overline{\nu }$	no isolated leptons
	$E_{missing}>50$ GeV
$b\overline{b}q\overline{q}q\overline{q}$	no isolated leptons
	GeV
$c\overline{c}q\overline{q}{l}^{+}\nu$	only 1 isolated lepton
$c\overline{c}q\overline{q}q\overline{q}$	no isolated leptons
	GeV

Table 4. Table 4: First selection level of the b ​ b ¯ ​ l + ​ l − ​ ν ​ ν ¯ b ¯ 𝑏 superscript 𝑙 superscript 𝑙 𝜈 ¯ 𝜈 {\mathrm{b}\bar{b}l^{+}l^{-}\nu\bar{\nu}} final state: the number of events selected in data and the SM expectations after the first selection level for each sample and centre-of-mass energy are shown.

$\sqrt{s}$ (GeV)	data (SM expectation $\pm$ statistical error)
	$e$ sample	$\mu$ sample	no-id sample
196	2 (2.6$\pm$0.3)	1 (2.9$\pm$0.3)	47 (35.9$\pm$1.4)
200	3 (2.5$\pm$0.4)	4 (3.4$\pm$0.4)	30 (37.4$\pm$1.4)
202	2 (1.3$\pm$0.2)	1 (1.7$\pm$0.2)	20 (18.7$\pm$0.7)
205	5 (2.5$\pm$0.4)	3 (3.0$\pm$0.4)	35 (36.2$\pm$1.4)
207	3 (2.3$\pm$0.4)	3 (3.1$\pm$0.4)	45 (35.1$\pm$1.3)
206∗	1 (1.9$\pm$0.3)	2 (2.6$\pm$0.2)	31 (27.6$\pm$1.0)
total	16 (13.2$\pm$0.8)	14 (16.7$\pm$0.8)	208 (191.0$\pm$3.0)

Table 5. Table 5: First selection level of the b ​ b ¯ ​ q ​ q ¯ ​ ν ​ ν ¯ b ¯ 𝑏 𝑞 ¯ 𝑞 𝜈 ¯ 𝜈 {\mathrm{b}\bar{b}q\bar{q}\nu\bar{\nu}} final state: the number of events selected in data and the SM expectation for each centre-of-mass energy are shown.

$\sqrt{s}$ (GeV)	data (SM expectation $\pm$ statistical error)
196	123 (106.3$\pm$4.0)
200	111 (104.8$\pm$4.0)
202	50 (49.8$\pm$1.9)
205	88 (94.2$\pm$3.7)
207	99 (91.2$\pm$3.6)
206∗	62 (65.7$\pm$2.6)
total	533 (511.7$\pm$8.3)

Table 6. Table 6: First selection level of the b ​ b ¯ ​ q ​ q ¯ ​ q ​ q ¯ b ¯ 𝑏 𝑞 ¯ 𝑞 𝑞 ¯ 𝑞 {\mathrm{b}\bar{b}q\bar{q}q\bar{q}} and c ​ c ¯ ​ q ​ q ¯ ​ q ​ q ¯ c ¯ 𝑐 𝑞 ¯ 𝑞 𝑞 ¯ 𝑞 {\mathrm{c}\bar{c}q\bar{q}q\bar{q}} final states: the number of events selected in data and the SM expectations for each centre-of-mass energy are shown.

$\sqrt{s}$ (GeV)	data (SM expectation $\pm$ statistical error)
196	349 (326.7$\pm$5.3)
200	347 (342.1$\pm$5.5)
202	165 (162.1$\pm$2.6)
205	322 (319.0$\pm$5.2)
207	287 (307.6$\pm$5.0)
206∗	192 (215.8$\pm$3.6)
total	1662 (1673.9$\pm$11.4)

Table 7. Table 7: First selection level of the c ​ c ¯ ​ q ​ q ¯ ​ l + ​ ν c ¯ 𝑐 𝑞 ¯ 𝑞 superscript 𝑙 𝜈 {\mathrm{c}\bar{c}q\bar{q}l^{+}\nu} final state: the number of events selected in data and the SM expectations for each sample and centre-of-mass energy are shown.

$\sqrt{s}$ (GeV)	data (SM expectation $\pm$ statistical error)
	$e$	$\mu$	no-id
196	65 (51.1$\pm$1.4)	53 (56.1$\pm$1.5)	38 (34.4$\pm$1.4)
200	54 (58.1$\pm$1.7)	63 (59.9$\pm$1.6)	40 (35.0$\pm$1.4)
202	30 (27.8$\pm$0.8)	21 (28.4$\pm$0.8)	13 (16.9$\pm$0.7)
205	56 (50.8$\pm$1.5)	66 (53.6$\pm$1.5)	32 (33.3$\pm$1.4)
207	53 (53.8$\pm$1.6)	48 (57.2$\pm$1.6)	35 (33.8$\pm$1.4)
206∗	31 (37.2$\pm$1.4)	42 (39.3$\pm$1.1)	21 (23.4$\pm$1.0)
total	289 (278.8$\pm$3.5)	293 (294.5$\pm$3.4)	179 (176.8 $\pm$ 2.8)

Table 8. Table 8: Summary of the total number of selected data events and SM expectations for the studied final states after the final selection (first selection level for b ​ b ¯ ​ l + ​ l − ​ ν ​ ν ¯ b ¯ 𝑏 superscript 𝑙 superscript 𝑙 𝜈 ¯ 𝜈 {\mathrm{b}\bar{b}l^{+}l^{-}\nu\bar{\nu}} ). The corresponding background composition and signal efficiencies for m b ′ = 100 subscript 𝑚 superscript b ′ 100 m_{\rm b^{\prime}}=100  GeV / c 2 absent superscript 𝑐 2 /c^{2} and s = 205 𝑠 205 \sqrt{s}=205  GeV are also shown.

		data	background				signal
final state		(SM $\pm$ stat. error)	composition (%)				efficiency (%)
			$q\overline{q}$	$\mathrm{WW}$	$\mathrm{ZZ}$	$\gamma \gamma$
$b\overline{b}{l}^{+}{l}^{-}\nu \overline{\nu }$	e sample	16 (13.2$\pm$0.8)	16	16	68	0	35.1$\pm$2.6
(first selection	$\mu$ sample	14 (16.7$\pm$0.8)	0	10	90	0	53.4$\pm$2.7
level)	aaa no-id sample	208 (191.0$\pm$3.0)	8	80	12	0	12.3$\pm$1.0
$b\overline{b}q\overline{q}\nu \overline{\nu }$		533 (511.7$\pm$8.3)	76	17	2	5	57.6$\pm$1.7
$b\overline{b}q\overline{q}q\overline{q}$		1662 (1673.9$\pm$11.4)	35	65	0	0	66.0$\pm$1.5
	e sample	289 (278.8$\pm$3.5)	7	82	11	0	45.3$\pm$2.7
$c\overline{c}q\overline{q}{l}^{+}\nu$	$\mu$ sample	293 (294.5$\pm$3.4)	2	97	1	0	56.4$\pm$2.7
	a no-id sample	179 (176.8$\pm$2.8)	9	84	7	0	5.3$\pm$0.7
	no lepton sample	533 (511.7$\pm$8.3)	76	17	2	5	8.9$\pm$0.9
$c\overline{c}q\overline{q}q\overline{q}$		1662 (1673.9$\pm$11.4)	35	65	0	0	67.3$\pm$1.5

Full text

CERN–PH-EP/2006-023

20 June 2006

DELPHI Collaboration

Abstract

A search for the pair production of fourth generation $b^{\prime}$-quarks was performed using data taken by the DELPHI detector at LEP-II. The analysed data were collected at centre-of-mass energies ranging from 196 to 209 GeV, corresponding to an integrated luminosity of 420 pb*-1*. No evidence for a signal was found. Upper limits on $BR({\mathrm{b}^{\prime}\to bZ})$ and $BR({\mathrm{b}^{\prime}\to cW})$ were obtained for ${\mathrm{b}^{\prime}}$ masses ranging from 96 to 103 GeV$/c^{2}$. These limits, together with the theoretical branching ratios predicted by a sequential four generations model, were used to constrain the value of $R_{CKM}=|\frac{V_{\mathrm{c}b^{\prime}}}{V_{\mathrm{t}b^{\prime}}V_{\mathrm{t}b}}|$, where $V_{\mathrm{c}b^{\prime}}$, $V_{\mathrm{t}b^{\prime}}$ and $V_{\mathrm{t}b}$ are elements of the extended CKM matrix.

(Accepted by Eur. Phys. J. C)

J.Abdallah${}^{\ref{LPNHE}}$, P.Abreu${}^{\ref{LIP}}$, W.Adam${}^{\ref{VIENNA}}$, P.Adzic${}^{\ref{DEMOKRITOS}}$, T.Albrecht${}^{\ref{KARLSRUHE}}$, R.Alemany-Fernandez${}^{\ref{CERN}}$, T.Allmendinger${}^{\ref{KARLSRUHE}}$, P.P.Allport${}^{\ref{LIVERPOOL}}$, U.Amaldi${}^{\ref{MILANO2}}$, N.Amapane${}^{\ref{TORINO}}$, S.Amato${}^{\ref{UFRJ}}$, E.Anashkin${}^{\ref{PADOVA}}$, A.Andreazza${}^{\ref{MILANO}}$, S.Andringa${}^{\ref{LIP}}$, N.Anjos${}^{\ref{LIP}}$, P.Antilogus${}^{\ref{LPNHE}}$, W-D.Apel${}^{\ref{KARLSRUHE}}$, Y.Arnoud${}^{\ref{GRENOBLE}}$, S.Ask${}^{\ref{LUND}}$, B.Asman${}^{\ref{STOCKHOLM}}$, J.E.Augustin${}^{\ref{LPNHE}}$, A.Augustinus${}^{\ref{CERN}}$, P.Baillon${}^{\ref{CERN}}$, A.Ballestrero${}^{\ref{TORINOTH}}$, P.Bambade${}^{\ref{LAL}}$, R.Barbier${}^{\ref{LYON}}$, D.Bardin${}^{\ref{JINR}}$, G.J.Barker${}^{\ref{WARWICK}}$, A.Baroncelli${}^{\ref{ROMA3}}$, M.Battaglia${}^{\ref{CERN}}$, M.Baubillier${}^{\ref{LPNHE}}$, K-H.Becks${}^{\ref{WUPPERTAL}}$, M.Begalli${}^{\ref{BRASIL-IFUERJ}}$, A.Behrmann${}^{\ref{WUPPERTAL}}$, E.Ben-Haim${}^{\ref{LAL}}$, N.Benekos${}^{\ref{NTU-ATHENS}}$, A.Benvenuti${}^{\ref{BOLOGNA}}$, C.Berat${}^{\ref{GRENOBLE}}$, M.Berggren${}^{\ref{LPNHE}}$, L.Berntzon${}^{\ref{STOCKHOLM}}$, D.Bertrand${}^{\ref{BRUSSELS}}$, M.Besancon${}^{\ref{SACLAY}}$, N.Besson${}^{\ref{SACLAY}}$, D.Bloch${}^{\ref{CRN}}$, M.Blom${}^{\ref{NIKHEF}}$, M.Bluj${}^{\ref{WARSZAWA}}$, M.Bonesini${}^{\ref{MILANO2}}$, M.Boonekamp${}^{\ref{SACLAY}}$, P.S.L.Booth†${}^{\ref{LIVERPOOL}}$, G.Borisov${}^{\ref{LANCASTER}}$, O.Botner${}^{\ref{UPPSALA}}$, B.Bouquet${}^{\ref{LAL}}$, T.J.V.Bowcock${}^{\ref{LIVERPOOL}}$, I.Boyko${}^{\ref{JINR}}$, M.Bracko${}^{\ref{SLOVENIJA1}}$, R.Brenner${}^{\ref{UPPSALA}}$, E.Brodet${}^{\ref{OXFORD}}$, P.Bruckman${}^{\ref{KRAKOW1}}$, J.M.Brunet${}^{\ref{CDF}}$, B.Buschbeck${}^{\ref{VIENNA}}$, P.Buschmann${}^{\ref{WUPPERTAL}}$, M.Calvi${}^{\ref{MILANO2}}$, T.Camporesi${}^{\ref{CERN}}$, V.Canale${}^{\ref{ROMA2}}$, F.Carena${}^{\ref{CERN}}$, N.Castro${}^{\ref{LIP}}$, F.Cavallo${}^{\ref{BOLOGNA}}$, M.Chapkin${}^{\ref{SERPUKHOV}}$, Ph.Charpentier${}^{\ref{CERN}}$, P.Checchia${}^{\ref{PADOVA}}$, R.Chierici${}^{\ref{CERN}}$, P.Chliapnikov${}^{\ref{SERPUKHOV}}$, J.Chudoba${}^{\ref{CERN}}$, S.U.Chung${}^{\ref{CERN}}$, K.Cieslik${}^{\ref{KRAKOW1}}$, P.Collins${}^{\ref{CERN}}$, R.Contri${}^{\ref{GENOVA}}$, G.Cosme${}^{\ref{LAL}}$, F.Cossutti${}^{\ref{TRIESTE}}$, M.J.Costa${}^{\ref{VALENCIA}}$, D.Crennell${}^{\ref{RAL}}$, J.Cuevas${}^{\ref{OVIEDO}}$, J.D’Hondt${}^{\ref{BRUSSELS}}$, J.Dalmau${}^{\ref{STOCKHOLM}}$, T.da Silva${}^{\ref{UFRJ}}$, W.Da Silva${}^{\ref{LPNHE}}$, G.Della Ricca${}^{\ref{TRIESTE}}$, A.De Angelis${}^{\ref{UDINE}}$, W.De Boer${}^{\ref{KARLSRUHE}}$, C.De Clercq${}^{\ref{BRUSSELS}}$, B.De Lotto${}^{\ref{UDINE}}$, N.De Maria${}^{\ref{TORINO}}$, A.De Min${}^{\ref{PADOVA}}$, L.de Paula${}^{\ref{UFRJ}}$, L.Di Ciaccio${}^{\ref{ROMA2}}$, A.Di Simone${}^{\ref{ROMA3}}$, K.Doroba${}^{\ref{WARSZAWA}}$, J.Drees${}^{\ref{WUPPERTAL},\ref{CERN}}$, G.Eigen${}^{\ref{BERGEN}}$, T.Ekelof${}^{\ref{UPPSALA}}$, M.Ellert${}^{\ref{UPPSALA}}$, M.Elsing${}^{\ref{CERN}}$, M.C.Espirito Santo${}^{\ref{LIP}}$, G.Fanourakis${}^{\ref{DEMOKRITOS}}$, D.Fassouliotis${}^{\ref{DEMOKRITOS},\ref{ATHENS}}$, M.Feindt${}^{\ref{KARLSRUHE}}$, J.Fernandez${}^{\ref{SANTANDER}}$, A.Ferrer${}^{\ref{VALENCIA}}$, F.Ferro${}^{\ref{GENOVA}}$, U.Flagmeyer${}^{\ref{WUPPERTAL}}$, H.Foeth${}^{\ref{CERN}}$, E.Fokitis${}^{\ref{NTU-ATHENS}}$, F.Fulda-Quenzer${}^{\ref{LAL}}$, J.Fuster${}^{\ref{VALENCIA}}$, M.Gandelman${}^{\ref{UFRJ}}$, C.Garcia${}^{\ref{VALENCIA}}$, Ph.Gavillet${}^{\ref{CERN}}$, E.Gazis${}^{\ref{NTU-ATHENS}}$, R.Gokieli${}^{\ref{CERN},\ref{WARSZAWA}}$, B.Golob${}^{\ref{SLOVENIJA1},\ref{SLOVENIJA3}}$, G.Gomez-Ceballos${}^{\ref{SANTANDER}}$, P.Goncalves${}^{\ref{LIP}}$, E.Graziani${}^{\ref{ROMA3}}$, G.Grosdidier${}^{\ref{LAL}}$, K.Grzelak${}^{\ref{WARSZAWA}}$, J.Guy${}^{\ref{RAL}}$, C.Haag${}^{\ref{KARLSRUHE}}$, A.Hallgren${}^{\ref{UPPSALA}}$, K.Hamacher${}^{\ref{WUPPERTAL}}$, K.Hamilton${}^{\ref{OXFORD}}$, S.Haug${}^{\ref{OSLO}}$, F.Hauler${}^{\ref{KARLSRUHE}}$, V.Hedberg${}^{\ref{LUND}}$, M.Hennecke${}^{\ref{KARLSRUHE}}$, H.Herr†${}^{\ref{CERN}}$, J.Hoffman${}^{\ref{WARSZAWA}}$, S-O.Holmgren${}^{\ref{STOCKHOLM}}$, P.J.Holt${}^{\ref{CERN}}$, M.A.Houlden${}^{\ref{LIVERPOOL}}$, J.N.Jackson${}^{\ref{LIVERPOOL}}$, G.Jarlskog${}^{\ref{LUND}}$, P.Jarry${}^{\ref{SACLAY}}$, D.Jeans${}^{\ref{OXFORD}}$, E.K.Johansson${}^{\ref{STOCKHOLM}}$, P.D.Johansson${}^{\ref{STOCKHOLM}}$, P.Jonsson${}^{\ref{LYON}}$, C.Joram${}^{\ref{CERN}}$, L.Jungermann${}^{\ref{KARLSRUHE}}$, F.Kapusta${}^{\ref{LPNHE}}$, S.Katsanevas${}^{\ref{LYON}}$, E.Katsoufis${}^{\ref{NTU-ATHENS}}$, G.Kernel${}^{\ref{SLOVENIJA1}}$, B.P.Kersevan${}^{\ref{SLOVENIJA1},\ref{SLOVENIJA3}}$, U.Kerzel${}^{\ref{KARLSRUHE}}$, B.T.King${}^{\ref{LIVERPOOL}}$, N.J.Kjaer${}^{\ref{CERN}}$, P.Kluit${}^{\ref{NIKHEF}}$, P.Kokkinias${}^{\ref{DEMOKRITOS}}$, C.Kourkoumelis${}^{\ref{ATHENS}}$, O.Kouznetsov${}^{\ref{JINR}}$, Z.Krumstein${}^{\ref{JINR}}$, M.Kucharczyk${}^{\ref{KRAKOW1}}$, J.Lamsa${}^{\ref{AMES}}$, G.Leder${}^{\ref{VIENNA}}$, F.Ledroit${}^{\ref{GRENOBLE}}$, L.Leinonen${}^{\ref{STOCKHOLM}}$, R.Leitner${}^{\ref{NC}}$, J.Lemonne${}^{\ref{BRUSSELS}}$, V.Lepeltier${}^{\ref{LAL}}$, T.Lesiak${}^{\ref{KRAKOW1}}$, W.Liebig${}^{\ref{WUPPERTAL}}$, D.Liko${}^{\ref{VIENNA}}$, A.Lipniacka${}^{\ref{STOCKHOLM}}$, J.H.Lopes${}^{\ref{UFRJ}}$, J.M.Lopez${}^{\ref{OVIEDO}}$, D.Loukas${}^{\ref{DEMOKRITOS}}$, P.Lutz${}^{\ref{SACLAY}}$, L.Lyons${}^{\ref{OXFORD}}$, J.MacNaughton${}^{\ref{VIENNA}}$, A.Malek${}^{\ref{WUPPERTAL}}$, S.Maltezos${}^{\ref{NTU-ATHENS}}$, F.Mandl${}^{\ref{VIENNA}}$, J.Marco${}^{\ref{SANTANDER}}$, R.Marco${}^{\ref{SANTANDER}}$, B.Marechal${}^{\ref{UFRJ}}$, M.Margoni${}^{\ref{PADOVA}}$, J-C.Marin${}^{\ref{CERN}}$, C.Mariotti${}^{\ref{CERN}}$, A.Markou${}^{\ref{DEMOKRITOS}}$, C.Martinez-Rivero${}^{\ref{SANTANDER}}$, J.Masik${}^{\ref{FZU}}$, N.Mastroyiannopoulos${}^{\ref{DEMOKRITOS}}$, F.Matorras${}^{\ref{SANTANDER}}$, C.Matteuzzi${}^{\ref{MILANO2}}$, F.Mazzucato${}^{\ref{PADOVA}}$, M.Mazzucato${}^{\ref{PADOVA}}$, R.Mc Nulty${}^{\ref{LIVERPOOL}}$, C.Meroni${}^{\ref{MILANO}}$, E.Migliore${}^{\ref{TORINO}}$, W.Mitaroff${}^{\ref{VIENNA}}$, U.Mjoernmark${}^{\ref{LUND}}$, T.Moa${}^{\ref{STOCKHOLM}}$, M.Moch${}^{\ref{KARLSRUHE}}$, K.Moenig${}^{\ref{CERN},\ref{DESY}}$, R.Monge${}^{\ref{GENOVA}}$, J.Montenegro${}^{\ref{NIKHEF}}$, D.Moraes${}^{\ref{UFRJ}}$, S.Moreno${}^{\ref{LIP}}$, P.Morettini${}^{\ref{GENOVA}}$, U.Mueller${}^{\ref{WUPPERTAL}}$, K.Muenich${}^{\ref{WUPPERTAL}}$, M.Mulders${}^{\ref{NIKHEF}}$, L.Mundim${}^{\ref{BRASIL-IFUERJ}}$, W.Murray${}^{\ref{RAL}}$, B.Muryn${}^{\ref{KRAKOW2}}$, G.Myatt${}^{\ref{OXFORD}}$, T.Myklebust${}^{\ref{OSLO}}$, M.Nassiakou${}^{\ref{DEMOKRITOS}}$, F.Navarria${}^{\ref{BOLOGNA}}$, K.Nawrocki${}^{\ref{WARSZAWA}}$, R.Nicolaidou${}^{\ref{SACLAY}}$, M.Nikolenko${}^{\ref{JINR},\ref{CRN}}$, A.Oblakowska-Mucha${}^{\ref{KRAKOW2}}$, V.Obraztsov${}^{\ref{SERPUKHOV}}$, O.Oliveira${}^{\ref{LIP}}$, S.M.Oliveira${}^{\ref{LIP}}$, A.Olshevski${}^{\ref{JINR}}$, A.Onofre${}^{\ref{LIP}}$, R.Orava${}^{\ref{HELSINKI}}$, K.Osterberg${}^{\ref{HELSINKI}}$, A.Ouraou${}^{\ref{SACLAY}}$, A.Oyanguren${}^{\ref{VALENCIA}}$, M.Paganoni${}^{\ref{MILANO2}}$, S.Paiano${}^{\ref{BOLOGNA}}$, J.P.Palacios${}^{\ref{LIVERPOOL}}$, H.Palka${}^{\ref{KRAKOW1}}$, Th.D.Papadopoulou${}^{\ref{NTU-ATHENS}}$, L.Pape${}^{\ref{CERN}}$, C.Parkes${}^{\ref{GLASGOW}}$, F.Parodi${}^{\ref{GENOVA}}$, U.Parzefall${}^{\ref{CERN}}$, A.Passeri${}^{\ref{ROMA3}}$, O.Passon${}^{\ref{WUPPERTAL}}$, L.Peralta${}^{\ref{LIP}}$, V.Perepelitsa${}^{\ref{VALENCIA}}$, A.Perrotta${}^{\ref{BOLOGNA}}$, A.Petrolini${}^{\ref{GENOVA}}$, J.Piedra${}^{\ref{SANTANDER}}$, L.Pieri${}^{\ref{ROMA3}}$, F.Pierre${}^{\ref{SACLAY}}$, M.Pimenta${}^{\ref{LIP}}$, E.Piotto${}^{\ref{CERN}}$, T.Podobnik${}^{\ref{SLOVENIJA1},\ref{SLOVENIJA3}}$, V.Poireau${}^{\ref{CERN}}$, M.E.Pol${}^{\ref{BRASIL-CBPF}}$, G.Polok${}^{\ref{KRAKOW1}}$, V.Pozdniakov${}^{\ref{JINR}}$, N.Pukhaeva${}^{\ref{JINR}}$, A.Pullia${}^{\ref{MILANO2}}$, J.Rames${}^{\ref{FZU}}$, A.Read${}^{\ref{OSLO}}$, P.Rebecchi${}^{\ref{CERN}}$, J.Rehn${}^{\ref{KARLSRUHE}}$, D.Reid${}^{\ref{NIKHEF}}$, R.Reinhardt${}^{\ref{WUPPERTAL}}$, P.Renton${}^{\ref{OXFORD}}$, F.Richard${}^{\ref{LAL}}$, J.Ridky${}^{\ref{FZU}}$, M.Rivero${}^{\ref{SANTANDER}}$, D.Rodriguez${}^{\ref{SANTANDER}}$, A.Romero${}^{\ref{TORINO}}$, P.Ronchese${}^{\ref{PADOVA}}$, P.Roudeau${}^{\ref{LAL}}$, T.Rovelli${}^{\ref{BOLOGNA}}$, V.Ruhlmann-Kleider${}^{\ref{SACLAY}}$, D.Ryabtchikov${}^{\ref{SERPUKHOV}}$, A.Sadovsky${}^{\ref{JINR}}$, L.Salmi${}^{\ref{HELSINKI}}$, J.Salt${}^{\ref{VALENCIA}}$, C.Sander${}^{\ref{KARLSRUHE}}$, R.Santos${}^{\ref{LIP}}$, A.Savoy-Navarro${}^{\ref{LPNHE}}$, U.Schwickerath${}^{\ref{CERN}}$, R.Sekulin${}^{\ref{RAL}}$, M.Siebel${}^{\ref{WUPPERTAL}}$, A.Sisakian${}^{\ref{JINR}}$, G.Smadja${}^{\ref{LYON}}$, O.Smirnova${}^{\ref{LUND}}$, A.Sokolov${}^{\ref{SERPUKHOV}}$, A.Sopczak${}^{\ref{LANCASTER}}$, R.Sosnowski${}^{\ref{WARSZAWA}}$, T.Spassov${}^{\ref{CERN}}$, M.Stanitzki${}^{\ref{KARLSRUHE}}$, A.Stocchi${}^{\ref{LAL}}$, J.Strauss${}^{\ref{VIENNA}}$, B.Stugu${}^{\ref{BERGEN}}$, M.Szczekowski${}^{\ref{WARSZAWA}}$, M.Szeptycka${}^{\ref{WARSZAWA}}$, T.Szumlak${}^{\ref{KRAKOW2}}$, T.Tabarelli${}^{\ref{MILANO2}}$, A.C.Taffard${}^{\ref{LIVERPOOL}}$, F.Tegenfeldt${}^{\ref{UPPSALA}}$, J.Timmermans${}^{\ref{NIKHEF}}$, L.Tkatchev${}^{\ref{JINR}}$, M.Tobin${}^{\ref{LIVERPOOL}}$, S.Todorovova${}^{\ref{FZU}}$, B.Tome${}^{\ref{LIP}}$, A.Tonazzo${}^{\ref{MILANO2}}$, P.Tortosa${}^{\ref{VALENCIA}}$, P.Travnicek${}^{\ref{FZU}}$, D.Treille${}^{\ref{CERN}}$, G.Tristram${}^{\ref{CDF}}$, M.Trochimczuk${}^{\ref{WARSZAWA}}$, C.Troncon${}^{\ref{MILANO}}$, M-L.Turluer${}^{\ref{SACLAY}}$, I.A.Tyapkin${}^{\ref{JINR}}$, P.Tyapkin${}^{\ref{JINR}}$, S.Tzamarias${}^{\ref{DEMOKRITOS}}$, V.Uvarov${}^{\ref{SERPUKHOV}}$, G.Valenti${}^{\ref{BOLOGNA}}$, P.Van Dam${}^{\ref{NIKHEF}}$, J.Van Eldik${}^{\ref{CERN}}$, N.van Remortel${}^{\ref{HELSINKI}}$, I.Van Vulpen${}^{\ref{CERN}}$, G.Vegni${}^{\ref{MILANO}}$, F.Veloso${}^{\ref{LIP}}$, W.Venus${}^{\ref{RAL}}$, P.Verdier${}^{\ref{LYON}}$, V.Verzi${}^{\ref{ROMA2}}$, D.Vilanova${}^{\ref{SACLAY}}$, L.Vitale${}^{\ref{TRIESTE}}$, V.Vrba${}^{\ref{FZU}}$, H.Wahlen${}^{\ref{WUPPERTAL}}$, A.J.Washbrook${}^{\ref{LIVERPOOL}}$, C.Weiser${}^{\ref{KARLSRUHE}}$, D.Wicke${}^{\ref{CERN}}$, J.Wickens${}^{\ref{BRUSSELS}}$, G.Wilkinson${}^{\ref{OXFORD}}$, M.Winter${}^{\ref{CRN}}$, M.Witek${}^{\ref{KRAKOW1}}$, O.Yushchenko${}^{\ref{SERPUKHOV}}$, A.Zalewska${}^{\ref{KRAKOW1}}$, P.Zalewski${}^{\ref{WARSZAWA}}$, D.Zavrtanik${}^{\ref{SLOVENIJA2}}$, V.Zhuravlov${}^{\ref{JINR}}$, N.I.Zimin${}^{\ref{JINR}}$, A.Zintchenko${}^{\ref{JINR}}$, M.Zupan${}^{\ref{DEMOKRITOS}}$

11footnotetext: Department of Physics and Astronomy, Iowa State University, Ames IA 50011-3160, USA 22footnotetext: IIHE, ULB-VUB, Pleinlaan 2, B-1050 Brussels, Belgium 33footnotetext: Physics Laboratory, University of Athens, Solonos Str. 104, GR-10680 Athens, Greece 44footnotetext: Department of Physics, University of Bergen, Allégaten 55, NO-5007 Bergen, Norway 55footnotetext: Dipartimento di Fisica, Università di Bologna and INFN, Via Irnerio 46, IT-40126 Bologna, Italy 66footnotetext: Centro Brasileiro de Pesquisas Físicas, rua Xavier Sigaud 150, BR-22290 Rio de Janeiro, Brazil 77footnotetext: Inst. de Física, Univ. Estadual do Rio de Janeiro, rua São Francisco Xavier 524, Rio de Janeiro, Brazil 88footnotetext: Collège de France, Lab. de Physique Corpusculaire, IN2P3-CNRS, FR-75231 Paris Cedex 05, France 99footnotetext: CERN, CH-1211 Geneva 23, Switzerland 1010footnotetext: Institut de Recherches Subatomiques, IN2P3 - CNRS/ULP - BP20, FR-67037 Strasbourg Cedex, France 1111footnotetext: Now at DESY-Zeuthen, Platanenallee 6, D-15735 Zeuthen, Germany 1212footnotetext: Institute of Nuclear Physics, N.C.S.R. Demokritos, P.O. Box 60228, GR-15310 Athens, Greece 1313footnotetext: FZU, Inst. of Phys. of the C.A.S. High Energy Physics Division, Na Slovance 2, CZ-180 40, Praha 8, Czech Republic 1414footnotetext: Dipartimento di Fisica, Università di Genova and INFN, Via Dodecaneso 33, IT-16146 Genova, Italy 1515footnotetext: Institut des Sciences Nucléaires, IN2P3-CNRS, Université de Grenoble 1, FR-38026 Grenoble Cedex, France 1616footnotetext: Helsinki Institute of Physics and Department of Physical Sciences, P.O. Box 64, FIN-00014 University of Helsinki,   Finland 1717footnotetext: Joint Institute for Nuclear Research, Dubna, Head Post Office, P.O. Box 79, RU-101 000 Moscow, Russian Federation 1818footnotetext: Institut für Experimentelle Kernphysik, Universität Karlsruhe, Postfach 6980, DE-76128 Karlsruhe, Germany 1919footnotetext: Institute of Nuclear Physics PAN,Ul. Radzikowskiego 152, PL-31142 Krakow, Poland 2020footnotetext: Faculty of Physics and Nuclear Techniques, University of Mining and Metallurgy, PL-30055 Krakow, Poland 2121footnotetext: Université de Paris-Sud, Lab. de l’Accélérateur Linéaire, IN2P3-CNRS, Bât. 200, FR-91405 Orsay Cedex, France 2222footnotetext: School of Physics and Chemistry, University of Lancaster, Lancaster LA1 4YB, UK 2323footnotetext: LIP, FCUL, IST, CFCUC - Av. Elias Garcia, 14-$1^{o}$, PT-1000 Lisboa Codex, Portugal 2424footnotetext: Department of Physics, University of Liverpool, P.O. Box 147, Liverpool L69 3BX, UK 2525footnotetext: Dept. of Physics and Astronomy, Kelvin Building, University of Glasgow, Glasgow G12 8QQ 2626footnotetext: LPNHE, IN2P3-CNRS, Univ. Paris VI et VII, Tour 33 (RdC), 4 place Jussieu, FR-75252 Paris Cedex 05, France 2727footnotetext: Department of Physics, University of Lund, Sölvegatan 14, SE-223 63 Lund, Sweden 2828footnotetext: Université Claude Bernard de Lyon, IPNL, IN2P3-CNRS, FR-69622 Villeurbanne Cedex, France 2929footnotetext: Dipartimento di Fisica, Università di Milano and INFN-MILANO, Via Celoria 16, IT-20133 Milan, Italy 3030footnotetext: Dipartimento di Fisica, Univ. di Milano-Bicocca and INFN-MILANO, Piazza della Scienza 3, IT-20126 Milan, Italy 3131footnotetext: IPNP of MFF, Charles Univ., Areal MFF, V Holesovickach 2, CZ-180 00, Praha 8, Czech Republic 3232footnotetext: NIKHEF, Postbus 41882, NL-1009 DB Amsterdam, The Netherlands 3333footnotetext: National Technical University, Physics Department, Zografou Campus, GR-15773 Athens, Greece 3434footnotetext: Physics Department, University of Oslo, Blindern, NO-0316 Oslo, Norway 3535footnotetext: Dpto. Fisica, Univ. Oviedo, Avda. Calvo Sotelo s/n, ES-33007 Oviedo, Spain 3636footnotetext: Department of Physics, University of Oxford, Keble Road, Oxford OX1 3RH, UK 3737footnotetext: Dipartimento di Fisica, Università di Padova and INFN, Via Marzolo 8, IT-35131 Padua, Italy 3838footnotetext: Rutherford Appleton Laboratory, Chilton, Didcot OX11 OQX, UK 3939footnotetext: Dipartimento di Fisica, Università di Roma II and INFN, Tor Vergata, IT-00173 Rome, Italy 4040footnotetext: Dipartimento di Fisica, Università di Roma III and INFN, Via della Vasca Navale 84, IT-00146 Rome, Italy 4141footnotetext: DAPNIA/Service de Physique des Particules, CEA-Saclay, FR-91191 Gif-sur-Yvette Cedex, France 4242footnotetext: Instituto de Fisica de Cantabria (CSIC-UC), Avda. los Castros s/n, ES-39006 Santander, Spain 4343footnotetext: Inst. for High Energy Physics, Serpukov P.O. Box 35, Protvino, (Moscow Region), Russian Federation 4444footnotetext: J. Stefan Institute, Jamova 39, SI-1000 Ljubljana, Slovenia 4545footnotetext: Laboratory for Astroparticle Physics, University of Nova Gorica, Kostanjeviska 16a, SI-5000 Nova Gorica, Slovenia 4646footnotetext: Department of Physics, University of Ljubljana, SI-1000 Ljubljana, Slovenia 4747footnotetext: Fysikum, Stockholm University, Box 6730, SE-113 85 Stockholm, Sweden 4848footnotetext: Dipartimento di Fisica Sperimentale, Università di Torino and INFN, Via P. Giuria 1, IT-10125 Turin, Italy 4949footnotetext: INFN,Sezione di Torino and Dipartimento di Fisica Teorica, Università di Torino, Via Giuria 1, IT-10125 Turin, Italy 5050footnotetext: Dipartimento di Fisica, Università di Trieste and INFN, Via A. Valerio 2, IT-34127 Trieste, Italy 5151footnotetext: Istituto di Fisica, Università di Udine and INFN, IT-33100 Udine, Italy 5252footnotetext: Univ. Federal do Rio de Janeiro, C.P. 68528 Cidade Univ., Ilha do Fundão BR-21945-970 Rio de Janeiro, Brazil 5353footnotetext: Department of Radiation Sciences, University of Uppsala, P.O. Box 535, SE-751 21 Uppsala, Sweden 5454footnotetext: IFIC, Valencia-CSIC, and D.F.A.M.N., U. de Valencia, Avda. Dr. Moliner 50, ES-46100 Burjassot (Valencia), Spain 5555footnotetext: Institut für Hochenergiephysik, Österr. Akad. d. Wissensch., Nikolsdorfergasse 18, AT-1050 Vienna, Austria 5656footnotetext: Inst. Nuclear Studies and University of Warsaw, Ul. Hoza 69, PL-00681 Warsaw, Poland 5757footnotetext: Now at University of Warwick, Coventry CV4 7AL, UK 5858footnotetext: Fachbereich Physik, University of Wuppertal, Postfach 100 127, DE-42097 Wuppertal, Germany

† deceased

1 Introduction

The Standard Model (SM), although in agreement with the available experimental data [1], leaves several open questions. In particular, the number of fermion generations and their mass spectrum are not predicted. The measurement of the ${\rm Z}$ decay widths [1] established that the number of light neutrino species ($m<m_{\rm Z}/2$, where $m_{\mathrm{Z}}$ is the ${\rm Z}$ boson mass) is equal to three. However, if a heavy neutrino or a neutrinoless extra generation exists, this bound does not exclude the possibility of extra generations of heavy quarks. Moreover the fit to the electroweak data [2] does not deteriorate with the inclusion of one extra heavy generation, if the new up and down-type quarks mass difference is not too large. It should be noticed however that in this fit no mixing of the extra families with the SM ones is assumed.

The subject of this paper is the search for the pair production of a fourth generation ${\mathrm{b}^{\prime}}$-quark at LEP-II: ${\mathrm{b}^{\prime}}$ production and decay are discussed in section 2; in section 3, the data sets and the Monte Carlo (MC) simulation are described; the analysis is discussed in section 4; the results and their interpretation within a sequential model are presented in sections 5 and 6, respectively.

2 $\boldsymbol{\rm b^{\prime}}$-quark production and decay

Extra generations of fermions are predicted in several SM extensions [3, 4]. In sequential models [5, 6, 7], a fourth generation of fermions carrying the same quantum numbers as the SM families is considered. In the quark sector, an up-type quark, ${\rm t^{\prime}}$, and a down-type quark, ${\rm b^{\prime}}$, are included. The corresponding $4\times 4$ extended Cabibbo-Kobayashi-Maskawa (CKM) matrix is unitary, approximately symmetric and almost diagonal. As CP-violation is not considered in the model, all the CKM elements are assumed to be real.

The ${\mathrm{b}^{\prime}}$-quark may decay via charged currents (CC) to $U{\rm W}$, with $U={\rm t^{\prime},t,c,u}$, or via flavour-changing neutral currents (FCNC) to $DX$, where $D={\rm b,s,d}$ and $X={\rm Z,H,\gamma,g}$ (Fig. 1). As in the SM, FCNC are absent at tree level, but can appear at one-loop level, due to CKM mixing. If the ${\mathrm{b}^{\prime}}$ is lighter than ${\rm t^{\prime}}$ and ${\rm t}$, the decays ${\rm b^{\prime}\to t^{\prime}W}$ and ${\rm b^{\prime}\to tW}$ are kinematically forbidden and the one-loop FCNC decays can be as important as the CC decays [6].

The analysis of the electroweak data [1] shows that the mass difference $|m_{\rm t^{\prime}}-m_{\rm b^{\prime}}|<60$ GeV$/c^{2}$ is consistent with the measurement of the $\rho$ parameter [3, 5]. In particular, when $m_{\rm Z}+m_{\rm b}<m_{\rm b^{\prime}}<m_{\rm H}+m_{\rm b}$, either ${\mathrm{b}^{\prime}\to cW}$ or ${\mathrm{b}^{\prime}\to bZ}$ decay tend to be dominant [5, 6, 7]. In this case, the partial widths of the CC and FCNC ${\mathrm{b}^{\prime}}$ decays depend mainly on $m_{\rm t^{\prime}}$, $m_{\rm b^{\prime}}$ and $R_{CKM}=|\frac{V_{\rm cb^{\prime}}}{V_{\rm tb^{\prime}}V_{\rm tb}}|$, where $V_{\rm cb^{\prime}}$, $V_{\rm tb^{\prime}}$ and $V_{\rm tb}$ are elements of the extended $4\times 4$ CKM matrix [7].

Limits on the mass of the ${\mathrm{b}^{\prime}}$-quark have been set previously at various accelerators. At LEP-I, all the experiments searched for ${\mathrm{b}^{\prime}}$ pair production (${\rm e^{+}e^{-}\to b^{\prime}\bar{b^{\prime}}}$), yielding a lower limit on the ${\mathrm{b}^{\prime}}$ mass of about $m_{\rm Z}/2$ [8]. At the Tevatron, both the D0[9] and CDF [10] experiments reported limits on $\sigma({\rm p\bar{p}}\to{\rm b^{\prime}}\bar{\rm b^{\prime}})\times BR({\rm b^{\prime}\to bX})^{2}$, where $BR$ is the branching ratio corresponding to the considered FCNC $b^{\prime}$ decay mode and $X=\gamma,{\rm Z}$. Assuming $BR({\rm b^{\prime}}\to{\rm bZ})=1$, CDF excluded the region $100<m_{\rm b^{\prime}}<199$ GeV$/c^{2}$. Although no dedicated analysis was performed for the ${\mathrm{b}^{\prime}\to cW}$ decay, the D0 limits on $\sigma({\rm p\bar{p}}\to{\rm t}\bar{\rm t})\times BR({\rm t\to cW})^{2}$ from Fig. 44 and Table XXXI of reference [11] can give a hint on the possible values for $BR({\mathrm{b}^{\prime}\to cW})$[12].

In the present analysis the on-shell FCNC (${\rm b^{\prime}\to bZ}$) and CC (${\rm b^{\prime}\to cW}$) decay modes were studied and consequently the mass range $96$ GeV$/c^{2}<m_{\rm b^{\prime}}<103$ GeV$/c^{2}$ was considered. This mass range is complementary to the one covered by CDF [10]. The mass range $m_{\rm W}+m_{\rm c}<m_{\rm b^{\prime}}<m_{\rm Z}+m_{\rm b}$ was not considered because in this region the evaluation of the branching ratios for the different $\rm b^{\prime}$ decays is particularly difficult from the theoretical point of view [7]. In the present analysis no assumptions on the $BR({\mathrm{b}^{\prime}\to bZ})$ and $BR({\mathrm{b}^{\prime}\to cW})$ in order to derive mass limits were made. Different final states, corresponding to the different ${\mathrm{b}^{\prime}}$ decay modes and subsequent decays of the ${\rm Z}$ and ${\rm W}$ bosons, were analysed.

3 Data samples and Monte Carlo simulation

The analysed data were collected with the DELPHI detector [13] during the years 1999 and 2000 in LEP-II runs at $\sqrt{s}=196-209$ GeV and correspond to an integrated luminosity of about 420 pb*-1*. The luminosity collected at each centre-of-mass energy is shown in Table 1. During the year 2000, an unrecoverable failure affected one sector of the central tracking detector (TPC), corresponding to 1/12 of its acceptance. The data collected during the year 2000 with the TPC fully operational were split into two energy bins, below and above $\sqrt{s}=206$ GeV, with $\langle\sqrt{s}\rangle=204.8$ GeV and $\langle\sqrt{s}\rangle=206.6$ GeV, respectively. The data collected with one sector of the TPC turned off were analysed separately and have $\langle\sqrt{s}\rangle=206.3$ GeV.

Signal samples were generated using a modified version of PYTHIA 6.200 [14]. Although PYTHIA does not provide FCNC decay channels for quarks, it was possible to activate them by modifying the decay products of an available channel. The angular distributions assumed for ${\mathrm{b}^{\prime}}$ pair production and decay were those predicted by the SM for any heavy down-type quark. Different samples, corresponding to ${\mathrm{b}^{\prime}}$ masses in the range between 96 and 103 GeV$/c^{2}$ and with a spacing of 1 GeV$/c^{2}$ were generated at each centre-of-mass energy. Specific Monte Carlo simulations (for both SM and signal processes) were produced for the period when one sector of the TPC was turned off.

The most relevant background processes for the present analyses are those leading to $WW$ or $ZZ$ bosons in the final state, i.e. four-fermion backgrounds. Radiation in these events can mimic the six-fermion final states for the signal. Additionally $q\bar{q}(\gamma)$ and Bhabha events can not be neglected since for signal final states with missing energy these backgrounds can become important. SM background processes were simulated at each centre-of-mass energy using several Monte Carlo generators. All the four-fermion final states (both neutral and charged currents) were generated with WPHACT [15], while the particular phase space regions of ${\rm e^{+}e^{-}\to e^{+}e^{-}f\bar{f}}$ referred to as $\gamma\gamma$ interactions were generated using PYTHIA [14]. The ${\rm qq(\gamma)}$ final state was generated with KK2F [16]. Bhabha events were generated with BHWIDE [17].

The generated signal and background events were passed through the detailed simulation of the DELPHI detector [13] and then processed with the same reconstruction and analysis programs as the data.

4 Description of the analyses

Pair production of ${\mathrm{b}^{\prime}}$-quarks was searched for in both the FCNC (${\rm b^{\prime}\to bZ}$) and CC (${\rm b^{\prime}\to cW}$) decay modes. The ${\mathrm{b}^{\prime}}$ decay modes and the subsequent decays of the gauge bosons (${\rm Z}$ or ${\rm W}$) lead to several different final states (Fig. 2). The final states considered and their branching ratios are shown in Table 2. The choice of the considered final states was done taking into account their signatures and BR. About 81% and 90% of the branching ratio to the FCNC and CC channels were covered, respectively. All final states include two jets originating from the low energy ${\rm b}$ (${\rm c}$) quarks present in the FCNC (CC) ${\rm b^{\prime}}$ decay modes. A common preselection was adopted, followed by a specific analysis for each of the final states (Table 2).

Events were preselected by requiring at least eight good charged-particle tracks and the visible energy measured at polar angles111In the standard DELPHI coordinate system, the positive $z$ axis is along the electron beam direction. The polar angle ($\theta$) is defined with respect to the $z$ axis. In this paper, polar angle ranges are always assumed to be symmetric with respect to the $\theta=90^{\circ}$ plane. above $20^{\circ}$, to be greater than $0.2\sqrt{s}$. Good charged-particle tracks were defined as those with a momentum above 0.2 GeV$/c$ and impact parameters in the transverse plane and along the beam direction below 4 cm and below 4 cm$/\sin\theta$, respectively.

The identification of muons relied on the association of charged particles to signals in the muon chambers and in the hadronic calorimeters and was provided by standard DELPHI algorithms [13]. The identification of electrons and photons was performed by combining information from the electromagnetic calorimeters and the tracking system. Radiation and interaction effects were taken into account by an angular clustering procedure around the main shower [18].

The search for isolated particles (charged leptons and photons) was done by constructing double cones oriented in the direction of charged-particle tracks or neutral energy deposits. The latter ones were defined as calorimetric energy deposits above $0.5$ GeV, not matched to charged-particle tracks and identified as photon candidates by the standard DELPHI algorithms [13, 18]. For charged leptons (photons), the energy in the region between the two cones, which had half-opening angles of $5^{\circ}$ and $25^{\circ}$ ($5^{\circ}$ and $15^{\circ}$), was required to be below 3 GeV (1 GeV), to ensure isolation. All the charged-particle tracks and neutral energy deposits inside the inner cone were associated to the isolated particle. Its energy was then re-evaluated as the sum of the energies inside the inner cone and was required to be above 5 GeV. For well identified leptons or photons [13, 18] the above requirements were weakened. In this case only the external cone was used (to ensure isolation) and its angle $\alpha$ was varied according to the energy of the lepton (photon) candidate, down to $2^{\circ}$ for $P_{\ell}\geq 70$ GeV/$c$ ($3^{\circ}$ for $P_{\gamma}\geq 90$ GeV/$c$), with the allowed energy inside the cone reduced by $\sin\alpha/\sin 25^{\circ}$ ($\sin\alpha/\sin 15^{\circ}$). Isolated leptons were required to have a momentum greater than 10 GeV$/c$ and a polar angle above $25^{\circ}$. Events with isolated photons were rejected.

All the events were clustered into two, four or six jets using the Durham jet algorithm [19], according to the number of jets expected in the signal in each of the final states, unless explicitly stated otherwise. Although two ${\rm b}$ jets are always present in the FCNC final states, they have a relatively low energy and b-tagging techniques [20] were not used.

Events were assigned to the different final states according to the number of isolated leptons and to the missing energy in the event, as detailed in Table 3. Within the same ${\mathrm{b}^{\prime}}$ decay channel, the different selections were designed to be mutually exclusive. For the final states involving charged leptons (${\mathrm{b}\bar{b}l^{+}l^{-}\nu\bar{\nu}}$ and ${\mathrm{c}\bar{c}q\bar{q}l^{+}\nu}$), events were divided into different samples according to the lepton flavour identification: ${\rm e}$ sample (well identified electrons), ${\rm\mu}$ sample (well identified muons) and no-id sample (leptons with unidentified flavour or two leptons identified with different flavours).

Specific analyses were then performed for each of the final states. The selection criteria for the ${\mathrm{b}\bar{b}q\bar{q}q\bar{q}}$ and ${\mathrm{c}\bar{c}q\bar{q}q\bar{q}}$ final states were the same. The ${\mathrm{b}\bar{b}l^{+}l^{-}\nu\bar{\nu}}$ final state has a very clean signature (two leptons with $m_{\rm l^{+}l^{-}}\sim m_{\rm Z}$, two low energy jets and missing mass close to $m_{\rm Z}$) and consequently a sequential cut analysis was adopted. For all the other final states, a sequential selection step was followed by a discriminant analysis. In this case, a signal likelihood (${\mathcal{L}}_{S}$) and a background likelihood (${\mathcal{L}}_{B}$) were assigned to each event, based on Probability Density Functions (PDF), built from the distributions of relevant physical variables. The discriminant variable was defined as $\ln({\mathcal{L}}_{S}/{\mathcal{L}}_{B})$.

4.1 The $\boldsymbol{\mathrm{b}\bar{b}l^{+}l^{-}\nu\bar{\nu}}$ final state

The FCNC ${\mathrm{b}\bar{b}l^{+}l^{-}\nu\bar{\nu}}$ final state events were preselected as described above, by requiring at least eight good charged-particle tracks, the visible energy measured at polar angles above $20^{\circ}$, to be greater than $0.2\sqrt{s}$ and at least one isolated lepton. Distributions of the relevant variables are shown in Fig. 3 for all the events assigned to this final state after the preselection. The event selection was performed in two levels. In the first one, events were required to have at least two leptons and an effective centre-of-mass energy [21], $\sqrt{s^{\prime}}$, below $0.95\sqrt{s}$. The particles other than the two leptons in the events were clustered into two jets and the Durham resolution variable in the transition from two jets to one jet222The Durham resolution variable is the minimum value of the scaled transverse momentum obtained in the transition from $n$ to $n-1$ jets [19] and will be represented by $y_{n\to n-1}$. was required to be greater than 0.002. The number of data events and the SM expectation after the first selection level is shown in Table 4. The background composition and the signal efficiencies at this level of selection for $m_{\rm b^{\prime}}=100$ GeV$/c^{2}$ and $\sqrt{s}=205$ GeV are given in Table 8. The efficiencies for the other relevant ${\mathrm{b}^{\prime}}$ masses and $\sqrt{s}$ values were found to be the same within errors. Data, SM expectation and signal distributions at this selection level are shown in Fig. 4.

In the final selection level the momentum of the more energetic (less energetic) jet was required to be below 30 GeV$/c$ (12.5 GeV$/c$). Events in the ${\rm e}$ and no-id samples had to have a missing energy greater than $0.4\sqrt{s}$. In the $\rm\mu$ sample events were required to have an angle between the two muons greater than 125∘. In the no-id sample, the angle between the two charged leptons had to be greater than 140∘ and $p_{mis}/E_{mis}<0.4$, where $p_{mis}$ and $E_{mis}$ are the missing momentum and energy, respectively. After the final selection, one data event was selected for an expected background of 1.5$\pm$0.7. This event belonged to the no-id sample and was collected at $\sqrt{s}=200$ GeV. The signal efficiencies for $m_{\rm b^{\prime}}=100$ GeV$/c^{2}$ and $\sqrt{s}=205$ GeV are $30.6\pm 2.5$% (${\rm e}$ sample), $48.6\pm 2.7$% ($\rm\mu$ sample) and $7.2\pm 0.8$% (no-id sample) and their variation with $m_{\rm b^{\prime}}$ and $\sqrt{s}$ was found to be negligible in the relevant range.

4.2 The $\boldsymbol{\mathrm{b}\bar{b}q\bar{q}\nu\bar{\nu}}$ final state

The FCNC ${\mathrm{b}\bar{b}q\bar{q}\nu\bar{\nu}}$ final state is characterised by the presence of four jets and a missing mass close to $m_{\rm Z}$. At least 20 good charged-particle tracks and $\sqrt{s^{\prime}}>0.5\sqrt{s}$ were required. Events were clustered into four jets. Monojet-like events were rejected by requiring $-\log_{10}(y_{2\to 1})<0.7$ ($y_{2\to 1}$ is the Durham resolution variable in the two to one jet transition). Furthermore, $-\log_{10}(y_{4\to 3})$ was required to be below 2.8 and the energy of the leading charged particle of the most energetic jet was required to be below $0.1\,\sqrt{s}$.

A kinematic fit imposing energy-momentum conservation and no missing energy was applied and the background-like events with $\chi^{2}/n.d.f.<6$ were rejected. The data, SM expectation and signal distributions of this variable are shown in Fig. 5. Table 5 summarizes the number of selected data events and the SM expectation. The background composition and the signal efficiency at this level of selection for $m_{\rm b^{\prime}}=100$ GeV$/c^{2}$ and $\sqrt{s}=205$ GeV are given in Table 8. The efficiencies for the other relevant ${\mathrm{b}^{\prime}}$ masses and $\sqrt{s}$ values were found to be the same within errors.

A discriminant selection was then performed using the following variables to build the PDFs:

• •

the missing mass;

• •

$A^{j_{1}j_{2}}_{cop}\times\mathrm{min}(\sin\theta_{j_{1}},\sin\theta_{j_{2}})$, where $A^{j_{1}j_{2}}_{cop}$ is the acoplanarity333The acoplanarity between two particles is defined as $|180^{\circ}-|\phi_{1}-\phi_{2}||$, where $\phi_{1,2}$ are the azimuthal angles of the two particles (in degrees). and $\theta_{j_{1},j_{2}}$ are the polar angles of the jets when forcing the events into two jets444While the signal is characterised by the presence of four jets in the final state, the two jets configuration is used mainly for background rejection.;

• •

the acollinearity between the two most energetic jets555The acollinearity between two particles is defined as $180^{\circ}-\alpha_{1,2}$, where $\alpha_{1,2}$ is the angle (in degrees) between those two particles. with the event particles clustered into four jets;

• •

the sum of the first and third Fox-Wolfram moments ($h_{1}+h_{3}$) [22];

• •

the polar angle of the missing momentum.

The data, SM expectation and signal distributions of these variables are shown in Fig. 6.

4.3 The $\boldsymbol{\mathrm{b}\bar{b}q\bar{q}q\bar{q}}$ final state

The FCNC ${\mathrm{b}\bar{b}q\bar{q}q\bar{q}}$ final state is characterised by the presence of six jets and a small missing energy. All the events were clustered into six jets and only those with at least 30 good charged-particle tracks were accepted. Moreover, events were required to have $\sqrt{s^{\prime}}>0.6\sqrt{s}$, $-\log_{10}(y_{2\to 1})<0.7$ and $-\log_{10}(y_{6\to 5})<3.6$. The number of selected data events and the expected background at this level are shown in Table 6. The background composition and the signal efficiency at this level of selection for $m_{\rm b^{\prime}}=100$ GeV$/c^{2}$ and $\sqrt{s}=205$ GeV are given in Table 8. The efficiencies for the other relevant ${\mathrm{b}^{\prime}}$ masses and $\sqrt{s}$ values were found to be the same within errors.

A discriminant selection was performed using the following variables to build the PDFs:

• •

the Durham resolution variable, $-\log_{10}(y_{4\to 3})$;

• •

the Durham resolution variable, $-\log_{10}(y_{5\to 4})$;

• •

the acollinearity between the two most energetic jets, with the event forced into four jets;

• •

the sum of the first and third Fox-Wolfram moments;

• •

the momentum of the most energetic jet;

• •

the angle between the two most energetic jets (with the events clustered into six jets).

The distributions of these variables are shown in Fig. 7 for data, SM expectation and signal.

4.4 The $\boldsymbol{\mathrm{c}\bar{c}q\bar{q}l^{+}\nu}$ final state

The signature of this CC final state is the presence of four jets (two of them having low energy), one isolated lepton and missing energy (originating from the ${\rm W\to l\bar{\nu}}$ decay). The events were accepted if they had at least 15 good charged-particle tracks. The event particles other than the identified lepton were clustered into four jets. Part of the ${\rm q\bar{q}}$ and $\gamma\gamma$ background was rejected by requiring $-\log_{10}(y_{2\to 1})<0.7$. Furthermore, there should be only one charged-particle track associated to the isolated lepton, and the leading charged particle of the most energetic jet was required to have a momentum below $0.1\sqrt{s}$. The number of selected data events and SM expectations at this level are summarized in Table 7. The background composition and the signal efficiencies at this level of selection for $m_{\rm b^{\prime}}=100$ GeV$/c^{2}$ and $\sqrt{s}=205$ GeV are given in Table 8. The efficiencies for the other relevant ${\mathrm{b}^{\prime}}$ masses and $\sqrt{s}$ values were found to be the same within errors.

The PDFs used to calculate the background and signal likelihoods were based on the following variables:

• •

the sum of the first and third Fox-Wolfram moments;

• •

the invariant mass of the two jets, with the event particles other than the identified lepton clustered into two jets;

• •

the Durham resolution variable, $-\log_{10}(y_{4\to 3})$;

• •

$\sum_{i}{|\vec{p}_{i}|}/\sqrt{s}$, where $\vec{p}_{i}$ are the momenta of the charged particles (excluding the lepton) in the same hemisphere as the lepton (the hemisphere is defined with respect to the lepton);

• •

the acollinearity between the two most energetic jets;

• •

the angle between the lepton and the missing momentum.

The data, SM expectation and signal distributions of these variables are shown in Fig. 8.

In order to improve the efficiency, events with no leptons seen in the detector were kept in a fourth sample. For this sample, the selection criteria of the ${\mathrm{b}\bar{b}q\bar{q}\nu\bar{\nu}}$ final state were applied and the same variables as in section 4.2 were used to build the PDFs. The signal efficiency after the first selection level for $m_{\rm b^{\prime}}=100$ GeV$/c^{2}$ and $\sqrt{s}=205$ GeV was $8.9\pm 0.9$%. The efficiencies for the other relevant ${\mathrm{b}^{\prime}}$ masses and $\sqrt{s}$ values were found to be the same within errors.

4.5 The $\boldsymbol{\mathrm{c}\bar{c}q\bar{q}q\bar{q}}$ final state

This final state is very similar to ${\mathrm{b}\bar{b}q\bar{q}q\bar{q}}$ (with slightly different kinematics due to the mass difference between the ${\rm Z}$ and the ${\rm W}$). The analysis described in section 4.3 was thus adopted. The number of selected events and the SM expectations can be found in Table 6. At this level, the signal efficiency for $m_{\rm b^{\prime}}=100$ GeV$/c^{2}$ and $\sqrt{s}=205$ GeV was $67.3\pm 1.5$%. The efficiencies for the other ${\mathrm{b}^{\prime}}$ masses and centre-of-mass energies were the same within errors. The PDFs were built using the same set of variables as in section 4.3.

5 Results

For all final states, a good agreement between data and SM expectation was found. The summary of the total number of selected data events, SM expectations, the corresponding background composition and the signal efficiencies for the studied final states are shown in Table 8. In the ${\mathrm{b}\bar{b}l^{+}l^{-}\nu\bar{\nu}}$ final state, one data event was retained after the final selection level, for a SM expectation of 1.5 $\pm$ 0.7 events. This event belonged to the no-id sample and was collected at $\sqrt{s}=200$ GeV. For all the other final states, discriminant analyses were used. In these cases, a discriminant variable, $\ln({\mathcal{L}}_{S}/{\mathcal{L}}_{B})$, was defined. The distributions of $\ln({\mathcal{L}}_{S}/{\mathcal{L}}_{B})$, for the different analysis channels are shown in Fig. 9. No evidence for a signal was found in any of the channels and the full information, i.e. event numbers and the shapes of the distributions of the discriminant variables were used to derive limits on $BR({\mathrm{b}^{\prime}\to bZ})$ and $BR({\mathrm{b}^{\prime}\to cW})$.

5.1 Limits on $\boldsymbol{BR({\mathrm{b}^{\prime}\to bZ})}$ and $\boldsymbol{BR({\mathrm{b}^{\prime}\to cW})}$

Upper limits on the product of the ${\rm e^{+}e^{-}\to b^{\prime}\bar{\rm b^{\prime}}}$ cross-section and the branching ratio as a function of the ${\mathrm{b}^{\prime}}$ mass were derived at 95% confidence level (CL) in each of the considered ${\mathrm{b}^{\prime}}$ decay modes (FCNC and CC), taking into account the values of the discriminant variables and their expected distributions for signal and background, the signal efficiencies and the data luminosities at the various centre-of-mass energies.

Assuming the SM cross-section for the pair production of heavy quarks at LEP[7, 14], these limits were converted into limits on the branching ratios corresponding to the ${\mathrm{b}^{\prime}\to bZ}$ and ${\mathrm{b}^{\prime}\to cW}$ decay modes. The modified frequentist likelihood ratio method [23] was used. The different final states and centre-of-mass energy bins were treated as independent channels. For each ${\mathrm{b}^{\prime}}$ mass only the channels with $\sqrt{s}>2\,m_{\rm b^{\prime}}$ were considered. In order to avoid some non-physical fluctuations of the distributions of the discriminant variables due to the limited statistics of the generated events, a smoothing algorithm was used. The median expected limit, i.e. the limit obtained if the SM background was the only contribution in data, was also computed. In Fig. 10 the observed and expected limits on $BR({\mathrm{b}^{\prime}\to bZ})$ and $BR({\mathrm{b}^{\prime}\to cW})$ are shown as a function of the ${\mathrm{b}^{\prime}}$ mass. The $1\sigma$ and $2\sigma$ bands around the expected limit are also shown. The observed and expected limits are statistically compatible. At 95% CL and for $m_{\rm b^{\prime}}=96$ GeV$/c^{2}$, the $BR({\mathrm{b}^{\prime}\to bZ})$ and $BR({\mathrm{b}^{\prime}\to cW})$ have to be below $51$% and $43$%, respectively. These limits were evaluated taking into account the systematic uncertainties, as explained in the next subsection.

The limits obtained for $BR({\mathrm{b}^{\prime}\to bZ})$ are compatible with those presented by CDF [10] for a ${\rm b^{\prime}}$ mass of 100 GeV$/c^{2}$. Below this mass, the DELPHI result is more sensitive and the CDF limit degrades rapidly. For higher ${\rm b^{\prime}}$ masses, the LEP-II kinematical limit is reached and the present analysis looses sensitivity.

5.2 Systematic uncertainties

The evaluation of the limits was performed taking into account systematic uncertainties, which affect the background estimation, the signal efficiency and the shape of the distributions used. The following systematic uncertainties were considered:

• •

SM cross-sections: uncertainties on the SM cross-sections translate into uncertainties on the expected number of background events. The overall uncertainty on the most relevant SM background processes for the present analyses is typically less than 2% [24], which leads to relative changes on the branching ratio limits below 6%;

• •

Signal generation: uncertainties on the final state quark hadronisation and fragmentation modelling were studied. The Lund symmetric fragmentation function was tested and compared with schemes where the ${\rm b}$ and ${\rm c}$ quark masses are taken into account [14]. This systematic error source was estimated to be of the order of 20% in the signal efficiency, by conservatively taking the maximum observed variation. The relative effect on the branching ratio limits is below 16%;

• •

Smoothing: the uncertainty associated to the discriminant variables smoothing was estimated by applying different smoothing algorithms. The smoothing procedure does not change the number of SM expected events or the signal efficiency, but may lead to differences in the shape of the discriminant variables. The relative effect of this uncertainty on the limits evaluation was found to be below 9%.

Further details on the evaluation of the systematic errors and the derivation of limits can be found in [25].

6 Constraints on $\boldsymbol{R_{CKM}}$

The branching ratios for the ${\mathrm{b}^{\prime}}$ decays can be computed within a four generations sequential model [5, 6, 7]. As discussed before, if the ${\mathrm{b}^{\prime}}$ is lighter than both the ${\rm t}$ and the ${\rm t^{\prime}}$ quarks and $m_{\rm Z}\,<m_{\rm b^{\prime}}<m_{\rm H}$, the main contributions to the ${\mathrm{b}^{\prime}}$ width are $BR({\mathrm{b}^{\prime}\to bZ})$ and $BR({\mathrm{b}^{\prime}\to cW})$ [7]. Using the unitarity of the CKM matrix, its approximate diagonality ($V_{\rm ub^{\prime}}\,V_{\rm ub}\approx\,0$) and taking $V_{\rm cb}\approx\,10^{-2}$ [12], the branching fractions can be written as a function of three variables: $R_{CKM}=|\frac{V_{\rm cb^{\prime}}}{V_{\rm tb^{\prime}}\,V_{\rm tb}}|$, $m_{\rm t^{\prime}}$ and $m_{\rm b^{\prime}}$ [5, 6, 7].

Fixing $m_{\rm t^{\prime}}-m_{\rm b^{\prime}}$, the limits on $BR({\mathrm{b}^{\prime}\to bZ})$ and $BR({\mathrm{b}^{\prime}\to cW})$ (Fig. 10) can be translated into 95% CL bounds on $R_{CKM}$ as a function of $m_{\rm b^{\prime}}$. Two extreme cases were considered: the almost degenerate case, with $m_{\rm t^{\prime}}-m_{\rm b^{\prime}}=1$ GeV$/c^{2}$, and the case in which the mass difference is close to the largest possible value, $m_{\rm t^{\prime}}-m_{\rm b^{\prime}}=50$ GeV$/c^{2}$ [3, 5]. The results are shown in Fig. 11 and Fig. 12. In the figures, the upper curve was obtained from the limit on $BR({\mathrm{b}^{\prime}\to cW})$, while the lower curve was obtained from the limit on $BR({\mathrm{b}^{\prime}\to bZ})$, which decreases with growing $m_{\rm t^{\prime}}$. This suppression is due to the GIM mechanism [26] as $m_{\rm t^{\prime}}$ approaches $m_{\rm t}$. On the other hand, as the ${\mathrm{b}^{\prime}}$ mass approaches the ${\rm bZ}$ threshold, the ${\rm b^{\prime}\to bg}$ decay dominates over ${\rm b^{\prime}\to bZ}$ [7] and the lower limit on $R_{CKM}$ becomes less stringent. The expected limits on $BR({\mathrm{b}^{\prime}\to bZ})$ did not allow to set exclusions for low values of $R_{CKM}$ and $m_{\rm t^{\prime}}-m_{\rm b^{\prime}}=1$ GeV$/c^{2}$ (see Fig. 11).

7 Conclusions

The data collected with the DELPHI detector at $\sqrt{s}=196-209$ GeV show no evidence for the pair production of ${\mathrm{b}^{\prime}}$-quarks with masses ranging from 96 to 103 GeV$/c^{2}$.

Assuming the SM cross-section for the pair production of heavy quarks at LEP, 95% CL upper limits on $BR({\mathrm{b}^{\prime}\to bZ})$ and $BR({\mathrm{b}^{\prime}\to cW})$ were obtained. It was shown that, at 95% CL and for $m_{\rm b^{\prime}}=96$ GeV$/c^{2}$, the $BR({\mathrm{b}^{\prime}\to bZ})$ and $BR({\mathrm{b}^{\prime}\to cW})$ have to be below $51$% and $43$%, respectively. The 95% CL upper limits on the branching ratios, combined with the predictions of the sequential fourth generation model, were used to exclude regions of the ($R_{CKM}$, $m_{\rm b^{\prime}}$) plane for two hypotheses of the $m_{\rm t^{\prime}}-m_{\rm b^{\prime}}$ mass difference. It was shown that, for $m_{\rm t^{\prime}}-m_{\rm b^{\prime}}=1~{}(50)$ GeV$/c^{2}$ and $96$ GeV$/c^{2}<m_{\rm b^{\prime}}<102$ GeV$/c^{2}$, $R_{CKM}$ is bounded by an upper limit of $3.8\times 10^{-3}$ ($1.2\times 10^{-3}$). For $m_{\rm b^{\prime}}=100$ GeV$/c^{2}$ and $m_{\rm t^{\prime}}-m_{\rm b^{\prime}}=50$ GeV$/c^{2}$, the CKM ratio was constrained to be in the range $4.6\times 10^{-4}<R_{CKM}<7.8\times 10^{-4}$.

Acknowledgements

We are greatly indebted to our technical collaborators, to the members of the CERN-SL Division for the excellent performance of the LEP collider, and to the funding agencies for their support in building and operating the DELPHI detector.

We acknowledge in particular the support of

Austrian Federal Ministry of Education, Science and Culture, GZ 616.364/2-III/2a/98,

FNRS–FWO, Flanders Institute to encourage scientific and technological research in the industry (IWT) and Belgian Federal Office for Scientific, Technical and Cultural affairs (OSTC), Belgium,

FINEP, CNPq, CAPES, FUJB and FAPERJ, Brazil,

Czech Ministry of Industry and Trade, GA CR 202/99/1362,

Commission of the European Communities (DG XII),

Direction des Sciences de la Mati$\grave{\mbox{\rm e}}$re, CEA, France,

Bundesministerium f$\ddot{\mbox{\rm u}}$r Bildung, Wissenschaft, Forschung und Technologie, Germany,

General Secretariat for Research and Technology, Greece,

National Science Foundation (NWO) and Foundation for Research on Matter (FOM), The Netherlands,

Norwegian Research Council,

State Committee for Scientific Research, Poland, SPUB-M/CERN/PO3/DZ296/2000, SPUB-M/CERN/PO3/DZ297/2000, 2P03B 104 19 and 2P03B 69 23(2002-2004)

FCT - Fundação para a Ciência e Tecnologia, Portugal,

Vedecka grantova agentura MS SR, Slovakia, Nr. 95/5195/134,

Ministry of Science and Technology of the Republic of Slovenia,

CICYT, Spain, AEN99-0950 and AEN99-0761,

The Swedish Research Council,

Particle Physics and Astronomy Research Council, UK,

Department of Energy, USA, DE-FG02-01ER41155,

EEC RTN contract HPRN-CT-00292-2002.