External Quality Assessment for the Detection of Chlamydia trachomatis in Urine Using Molecular Techniques in Belgium

TL;DR

This study evaluates the accuracy of molecular techniques for detecting Chlamydia trachomatis in urine in Belgium, highlighting areas for improvement.

Contribution

The study introduces and evaluates an external quality assessment program for Chlamydia detection in Belgium.

Findings

Overall accuracy of detection was 75.4% with high specificity (97.6%) but lower sensitivity (71.4%).

Low sensitivity (45.3%) was observed for low concentration samples and some methods failed to detect the Swedish variant.

Laboratory proficiency improved over time according to the study.

Abstract

Chlamydia trachomatis is a major cause of sexually transmitted bacterial disease worldwide. C. trachomatis is an intracellular bacterium and its growth in vitro requires cell culture facilities. The diagnosis is based on antigen detection and more recently on molecular nucleic acid amplification techniques (NAAT) that are considered fast, sensitive, and specific. In Belgium, External Quality Assessment (EQA) for the detection of C. trachomatis in urine by NAAT was introduced in 2008. From January 2008 to June 2012, nine surveys were organized. Fifty-eight laboratories participated in at least one survey. The EQA panels included positive and negative samples. The overall accuracy was 75.4%, the overall specificity was 97.6%, and the overall sensitivity was 71.4%. Two major issues were observed: the low sensitivity (45.3%) for the detection of low concentration samples and the incapacity of several methods to detect the Swedish variant of C. trachomatis. The reassuring point was that the overall proficiency of the Belgian laboratories tended to improve over time.

Figures

Full text

1. Introduction

Chlamydia trachomatis infection is the most prevalent sexually transmitted bacterial disease and is, therefore, a significant global health problem. It is estimated that 90 million cases occur annually worldwide [1]. The number of those infected is likely to be much higher because most of the infected people were asymptomatic [2]. In Belgium, the number of diagnosed cases was 3314 in 2010 with an incidence rate of 30.8/100 000 people [3].

C. trachomatis is a nonmotile obligate intracellular bacterium characterized by a unique biphasic developmental cycle [4]. Based on the antigenic reactivity of the OMP (Outer Membrane Proteins),* C. trachomatis* is currently divided into 18 serotypes. Serotypes A, B, Ba, and C are generally associated with blinding trachoma and serotypes D to K are responsible for causing nondisseminating sexually transmitted infections. These 12 serotypes (A, B, Ba, C and D–K) are all naturally restricted to infection of genital or ocular epithelial cells and have not been observed as invasive [5]. By contrast serotypes L1, L2, L2a, and L3 cause a number of invasive and systemic sexually transmitted infections normally found in the tropics, known as lymphogranuloma venereum (LGV) [6].

The type and anatomical site of specimen collection for laboratory diagnosis of* C. trachomatis* infection depend on both the clinical picture and the laboratory test selection [4]. Noninvasively collected specimens such as first-void urine and vulvogenital swab specimen are excellent for the diagnosis of* C. trachomatis* genital tract infection by nucleic acid amplification techniques (NAAT). Due to their high sensitivity and specificity, NAAT are the tests of choice for diagnosis of genital* C. trachomatis* infections in routine clinical laboratories. NAAT can be used to detect* C. trachomatis* without a pelvic examination or intrauteral swab specimen by testing self- or clinician-collected vaginal swab or urine [4]. In many evaluations, NAAT detected 20 to 30% more positive specimens than could be detected by non-NAAT technologies. Licensed NAAT for detection of* C. trachomatis* include (i) PCR-based methods either conventional PCR methods such as Roche Amplicor or the real time PCR methods such as Roche TaqMan (Roche Diagnostics, Basel, Switzerland) and the Abbott real time (TM) CT or CT/NG assay (Abbott, Abbott Park, IL, USA), (ii) Transcription Mediated Amplification (TMA) such as APTIMA (Gen-Probe Inc., San Diego, CA), and (iii) the strand displacement amplification (SDA) such as the BD ProbeTec method (Becton Dickinson and Company, Diagnostic Systems, Franklin Lakes, NJ).

In 2006, a new variant of* C. trachomatis* was described in Sweden presenting a 377 bp deletion of the plasmid DNA [7]. Since some NAAT are based on the detection plasmid specific DNA regions, some false negative results can occur [7].

In Belgium, the reimbursement by the social security insurance of the detection of microbes using molecular techniques by medical laboratories was specifically introduced into the legislation in 2008 [8]. The reimbursement was coupled to the obtaining of ISO15189 [9] accreditation and to the participation in External Quality Assessment (EQA). Since 2008, the Belgian Scientific Institute of Public Health (IPH) has organized the EQA for these laboratories including the detection of* C. trachomatis* in urine and swabs. The present paper describes the results obtained from 2008 to 2012 for the detection of* C. trachomatis* in urine using NAAT.

2. Material and Methods

2.1. The Samples

From 2008 to 2012, 9 EQA sample panels were provided to the participants. It means two panels per year (CTA and CTB) except for 2012 where only one panel was provided. These panels consisted of urine and simulated swabs samples. In this paper only urine samples were considered. The EQA samples (Table 1) were provided by Quality Control for Molecular Diagnostics (QCMD, Glasgow, Scotland). QCMD is accredited under the international standard ISO17043 [10] for the provision of EQA. The* C. trachomatis* strains used were either* C. trachomatis* LGV serovar L2 or* C. trachomatis* Swedish variant [7]. Samples were lyophilized and required reconstitution following the instructions manual provided with the samples. Laboratories were instructed to process the samples as routine urine or swab samples.

The samples varied in their amount of target* C. trachomatis* DNA. Samples could be negative meaning that no* C. trachomatis* DNA molecule was present.

2.2. The Participants

In Belgium, EQA is mandatory for the clinical biology laboratories [12]. In 2008, some parameters of molecular microbiology were introduced into the scope of the EQA scheme [13]. From 2008 to 2012, fifty-eight Belgian laboratories were registered yearly to the EQA for the detection of* C. trachomatis* using molecular techniques.

2.3. The Procedure

The registered laboratories received the EQA samples (Table 1) and were given around one month time to return their results to QCMD via the QCMD web page (http://www.qcmd.org/). Each participant possessed its own login and password to have access to their personal participant area. Information about the extraction method, the detection method, the testing results, and any encountered problems was entered online.

After the closure of the results return period, participants received an individual and an overall final report. At the end of a cycle, the Belgian participants received an annual report including their results for all the panels of the previous year.

2.4. Evaluation

Sample status is assigned by peer-group consensus based on the qualitative results returned by all participants in the full EQA program. It was not a measure of the “strength” of a positive sample nor was it technology-dependent and was used solely for the scoring of the EQA data. The rationale for the sample status was as follows.

• Frequently detected: more than 95% of datasets recorded the correct positive result.
• Detected: between 65 and 95% of the datasets recorded the correct positive result.
• Infrequently detected: less than 65% of the datasets recorded the correct positive result.
• Negative: a sample that does not contain the target produced an unequivocal negative result. A scoring system was established by QCMD for individual performance assessment (Table 2). For each correct answer the lab received a score of 0. A false positive result was scored as +3. A false negative result was scored as +1 for infrequently detected samples; +2 for detected samples; and +3 for frequently detected samples. Therefore, using this scoring system, the lowest score is the best. A “not determined” result was not scored. For IPH, a false result was considered a clinically relevant fault in two cases: a false positive result and a false negative result for a frequently detected sample. The participant encoding a clinically relevant fault is susceptible to receive an official claim from the IPH. This claim must be treated as a nonconformity in their quality management system.

2.5. The Methods

The different methods used by the Belgian laboratories during the surveys were listed in Table 3.

3. Results

3.1. Participants

Between January 2008 and June 2012, nine surveys were organized.

The number of participating laboratories ranged from 51 to 54 per survey. The percentage of responding laboratories ranged from 85 to 96%. The number of datasets returned to QCMD was always higher than the number of responding laboratories indicating that some laboratories introduced more than one dataset. Indeed, EQA is very often an occasion for the laboratories to validate new methods.

From 2008 to 2012, 58 different laboratories responded; among them 33 (56.9%) participated in 9 surveys, 5 (8.6%) in 8 surveys, 6 (10.3%) in 7 surveys, 1 (1.7%) in 6 surveys, 4 (6.9%) in 5 surveys, 2 (3.4%) in 4 surveys, 1 (1.7%) in 3 surveys, 3 (5.2%) in 2 surveys, and 3 (5.2%) in 1 survey.

3.2. Proficiency for Urine Samples

The overall number of Belgian results was 2917. The returned results were always qualitative (presence or absence) results. The number of correct results (accuracy) was 2198/2917 (75.4%). The number of incorrect results was 719/2917 (24.6%). Of the incorrect results, the number of false positive results was 6/719 (0.8%) and the number of false negative results was 693/719 (96.4%). The number of inhibition results was 20/719 (2.8%). The number of negative and frequently detected samples was 445 and 747, respectively. The number of clinically relevant faults was 61/1192 (5.1%) including 6/445 (1.3%) false positive and 55/747 (7.4%) false negative. The overall sensitivity and specificity were 71.4% (1764/2472) and 97.6% (434/445), respectively. When the yearly evolution was considered (Figure 1), a general increase both in sensitivity and in specificity was observed from 2008 to 2012. For the 2011-1 survey, a decrease in sensitivity was observed partially due to the presence of a sample with a very low copy number (CTADNA11-01).

The proficiency per sample is shown in Figure 2. The percentage of correct answers ranged widely from 2.1% for sample CTA08-02 to 100% for samples CTA10-02, CTA10-05, CTA10-06, CTB10-03, and CTA12-05.

The samples with regard to the amount of target genome present were divided into frequently detected, detected, infrequently detected, and negative sample status. The percentage of correct answers was 92.4% for frequently detected, 86.4% for detected, 45.3% for infrequently detected, and 97.5% for negative samples, respectively.

3.3. Comparison of the Detection Methods

When the detection methods used were considered (Table 4), the most frequently used methods were Roche Cobas TaqMan CT method (23.5%), the BD ProbeTec ET system (17.7%), and the Roche Cobas Amplicor CT/NG method (13.4%). The methods used in less than 5 series of results were not considered. The methods can be ranked regarding their accuracy results (the % of correct answers). The accuracy ranged from 64.1 (Roche Amplicor CT/NG) to 93.7% (Nanogen Chlamydia tr. Q-PCR Alert).

3.4. The Swedish Variant

During the nine surveys from 2008 to 2012, 8 samples (1 infrequently detected, 1 detected, and six frequently detected) consisted of the Swedish variant of* C. trachomatis*. The results indicated that only 307 out of 401 (76.5%) answers were correct. The study of the methods used to analyze these samples (Table 3) showed that some kits (Roche Amplicor and Roche Cobas Amplicor) were unable to detect this* C. trachomatis* Swedish variant.

3.5. The Scores of the Laboratories

The laboratories were evaluated by the attribution of a score (Table 2). When the IPH scores of the laboratories were considered over the time (Figure 3), the trend was a decrease in the scores indicating an increase in the proficiency of the Belgian laboratories.

4. Discussion

Chlamydia trachomatis is responsible for sexually transmitted disease. In Belgium, the number of diagnosed cases increased from 691 in 1997 to 3314 in 2010 [3].* C. trachomatis* is a fastidious bacterium and its growth can be problematic. Classically the diagnosis of* C. trachomatis* was based on the detection of specific antigens. Therefore, the introduction of molecular techniques for the detection of* C. trachomatis* in urine or swabs was a helpful progress in the diagnosis of* C. trachomatis* infection. Indeed, clinical evaluation of the NAAT has shown that they are more sensitive than culture and other methods including microscopy, antigen detection, and nucleic acid hybridization [4].

In Belgium, mandatory EQA for the detection of* C. trachomatis* by NAAT was introduced in 2008 [8]. The laboratories must participate in the External Quality Assessment (EQA) organized by the Scientific Institute of Public Health. In this context, since 2008, the IPH has organized the EQA for the detection of* C. trachomatis* in urine and swabs in collaboration with QCMD.

Between January 2008 and December 2012, nine surveys were organized. Only urine results are discussed in this paper since swabs results gave no additional information. For Belgium, 58 laboratories participated in at least one survey. These laboratories are clinical biology laboratories according to the Belgian legislation [12], representing about 30% of the total of the Belgian clinical biology laboratories. Among the IPH proposed EQA microbiological parameters for the detection by molecular techniques,* C. trachomatis* is the parameter for which the biggest number of laboratories was registered.

The Belgian participants represented around 28% of the total number of the participants in QCMD survey.

For* C. trachomatis* the encoded results were qualitative results although some NAAT methods allow quantification.

When the results were considered in relation to the type of samples, two major issues were observed. First, the sensitivity was low (below 50%) for the detection of infrequently detected samples. The results indicated that when the number of copies per vial was under 60, the detection power was low. It raises the question of the clinical cut-off. It seems clear that in case of a true infection the level of* C. trachomatis* is expected to be high.

Second, the sensitivity for the detection of the Swedish variant strain of* C. trachomatis* is also low (around 75%). It is particularly clear (Table 3) that some methods were unable to detect this variant. This is due to the fact that the Swedish variant has a 377 bp deletion in the cryptic plasmid [11]. Therefore detection kits such as the Roche (Cobas) Amplicor CT/NG kit that target the deleted region were unable to detect the Swedish variant [14, 15]. In our study, the Roche Cobas/Amplicor CT/NG kits were also unable to detect the Swedish variant even when present in large amounts. It should be noted that the insert of these kits contains the mention: “The Amplicor CT/NG Test or the Cobas Amplicor CT/NG test for* Chlamydia trachomatis* will not detect plasmid-free variants of* C. trachomatis*.” For the Roche Cobas TaqMan CT/NG kit, it is interesting to notice that, in 2008 and in the first survey of 2009, the kit was unable to detect the variant DNA but due to an improvement of the kit (v2.0), the further samples were well detected. Therefore, it is important that all the laboratories do not use the same method to detect* C. trachomatis* to minimize the fact that the presence of a new variant will be not detected. Moreover, the case of the Swedish variant underlines the fact that the EQA is a powerful means to detect the failing of kits or methods on certain samples.

The new Swedish variant (nvCT) represents 20 to 64% of the detected Chlamydia cases in Sweden [14]. Although the nvCT has been detected in Norway, Finland, and Denmark [16], only a few cases of nvCT have been reported outside the Nordic countries [17–19].

Among the most used methods, the Abbott real time PCR CT/NG and the Roche Cobas TaqMan CT/NG v2.0 gave the best results with an accuracy of 87.4 and 90.1%, respectively. It is not clear from the results that the homemade methods were less proficient than the commercial kits.

When the technologies used were compared, real time PCR methods gave better results than SDA methods or than conventional PCR methods. The TMA (Transcription Mediated Amplification) methodology was only used by one participant in two surveys and was not considered here.

Finally, when the proficiency over time was considered, a decrease in the penalty points scored was observed indicating an increase in proficiency. This is encouraging for the future and that fully justifies the need of EQA.

EQA allows comparing the proficiency of the laboratories but also of the methodologies. For a small country such as Belgium, it is also useful to participate in international surveys including other countries in order to be able to compare the result of a lab with other laboratories using the same methodology. It is particularly true for the methods that are not frequently used.

The major issue for the EQA is to have samples as close as possible to clinical samples. But these samples must also be homogeneous and stable. Moreover, the pre- and postanalytical process should also be assessed. For the preanalytical phase, the sampling and the transport conditions are important. The postanalytical evaluation is particularly important in case of low contaminated samples. It is not evident that all the laboratories will give the same answer to the clinician. The determination of an analytical and a clinical cut-off is sometimes required. Nevertheless, the improvement of the quality of the diagnosis is our priority and a powerful EQA is a major tool in this goal.

5. Conclusion

A correct diagnosis is closely related to powerful diagnostic tools. The proficiency of these methods can be evaluated using External Quality Assessment. In Belgium, for many years, the participation in EQA is mandatory for the medical laboratories. Moreover, for molecular testing, the ISO15189 accreditation is also required. The results of the EQA for the detection of* C. trachomatis* in urine using molecular methods revealed a low sensitivity (71.4%) but a good specificity (97.6%). The low sensitivity is mainly related to the lack of detection of the Swedish variant by several methods. Nevertheless, the situation improved with time indicating that the laboratories and the companies make effort to guaranty the best result.