External Validation of the JAKPOT Score for Diagnosing JAK2-Positive Erythrocytosis: A Retrospective Cohort Study

TL;DR

This study validates a new score to identify JAK2-positive erythrocytosis using blood counts and erythropoietin levels, reducing the need for genetic testing.

Contribution

The EPO-JAKPOT score combines low EPO and JAKPOT criteria to improve diagnostic accuracy for JAK2 mutations in erythrocytosis.

Findings

JAKPOT+ status had 88% sensitivity for detecting JAK2 mutations.

Combining low EPO with JAKPOT increased sensitivity to 95%.

Using EPO-JAKPOT could reduce molecular tests by 55% in similar patient groups.

Abstract

Background/Objectives: Erythrocytosis is a common laboratory abnormality affecting approximately 4% of males and 0.4% of females. The JAKPOT score was recently developed to differentiate primary from secondary erythrocytosis without molecular testing. JAKPOT+ patients meet any of the following criteria: erythrocytes > 6.45 × 1012/L, platelets > 350 × 109/L, or neutrophils > 6.2 × 109/L. We aimed to validate this score and identify predictors of JAK2-positive erythrocytosis in a retrospective cohort. Methods: We identified 213 patients (50 female, mean age 57 years) with undifferentiated erythrocytosis, serum erythropoietin (EPO) and JAK2 molecular testing (V617F or exon 12) at a tertiary care center in Hamilton, Canada, between 2017 and 2022. Charts were manually reviewed for laboratory data, comorbidities, demographics, and medications. We evaluated the diagnostic accuracy of EPO, JAKPOT, and a combination of low EPO and JAKPOT (EPO-JAKPOT) for predicting JAK2 mutant erythrocytosis. Multivariate logistic regression analysis was performed to detect predictors of JAK2 mutant erythrocytosis. Results: Forty patients (19%) had JAK2 mutations. Older age (p < 0.01), higher platelet count (p < 0.01), and lower EPO (p < 0.01) were associated with JAK2 mutant erythrocytosis in a multivariate analysis. JAKPOT+ status had a sensitivity of 0.88 (95% CI, 0.73–0.94). Combining low EPO or JAKPOT+ status into a new score (EPO-JAKPOT) increased sensitivity to 0.95 (95% CI, 0.83–0.98). Restricting JAK2 testing to only EPO-JAKPOT+ patients would have led to 55% fewer molecular tests in our cohort. Conclusions: The EPO-JAKPOT score shows promise in excluding JAK2 mutant erythrocytosis without molecular testing, but further prospective validation is warranted.

Figures

Full text

1. Introduction

Erythrocytosis, defined by the World Health Organization (WHO) as a hemoglobin (Hb) greater than 165 g/L for males or 160 g/L for females [1], is a common laboratory finding affecting up to 4% of males and 0.4% of females [2]. The large majority of such individuals have secondary erythrocytosis, whereas polycythemia vera (PV)—the most common cause of primary erythrocytosis—is relatively rare, with an incidence of only 0.84 cases per 100,000 individuals [3,4].

The hallmark laboratory criteria for diagnosing PV are an elevated Hb and/or hematocrit (Hct), low serum erythropoietin (EPO), mutated Janus Kinase 2 (JAK2) and a bone marrow biopsy showing hypercellularity for age, trilineage growth (panmyelosis) and pleomorphic, mature megakaryocytes [5]. In isolation, low serum EPO lacks sensitivity for PV, such that 80% of patients with PV have a normal EPO level [6]. Activating mutations in JAK2 are present in 98% of patients with polycythemia vera [7], but molecular genetic testing is relatively costly and is not universally available across clinical laboratories. Therefore, there is an unmet need for a reliable, cost-effective, and widely accessible strategy to differentiate between PV from secondary erythrocytosis.

Chin-Yee et al. (2023) [8] recently developed the JAKPOT score as a screening test for JAK2 mutant erythrocytosis. The JAKPOT prediction rule includes only three parameters from the complete blood count (CBC) in patients who have not been treated for suspected PV: (i) absolute red blood cells > 6.45 × 10^12^/L, (ii) platelets > 350 × 10^9^/L, and (iii) neutrophils > 6.2 × 10^9^/L. In the JAKPOT derivation study, the absence of all three criteria ruled out JAK2 mutant erythrocytosis with a sensitivity of 95–100% and a false negative rate of 0.4%, but the score has not been externally validated.

Our aim was to externally validate the JAKPOT score in a retrospective cohort of patients investigated for erythrocytosis at a tertiary care hospital. Our secondary aims were to identify predictors of JAK2 mutant erythrocytosis in our cohort.

2. Methods and Materials

2.1. Study Population

We conducted a retrospective cohort study at a 777-bed tertiary care hospital in Hamilton, Canada. Adult (>18 years of age) inpatients and outpatients with both an EPO level and JAK2 molecular testing performed between 31 December 2017 and 31 December 2022 were included. Patients were excluded if they were anemic (hemoglobin < 115 g/L) at the time of EPO testing, had incomplete records or were known to have donated blood or phlebotomized in the three months before testing. Recently phlebotomized patients were excluded because phlebotomy would confound the interpretation of the hemoglobin, hematocrit and EPO level. This study was approved by the Hamilton Integrated Research Ethics Board (#15535).

2.2. Data Extraction and Laboratory Assays

We reviewed electronic medical records for demographic data, comorbidities, laboratory parameters, imaging studies, molecular test results for JAK2 V617F or exon 12 mutations, pathology results, pulmonary function test results, and medications, including anticoagulants, antiplatelets, diuretics, testosterone, and sodium–glucose transporter-2 (SGLT2) inhibitors. We recorded laboratory parameters from the same day that JAK2 molecular testing was ordered; if unavailable, we recorded the most recent values obtained before molecular testing. We recorded information from clinical notes as close as possible to the date of EPO testing in a RedCAP database [9]. Data were extracted by JS, YM, RT, and SR. Eighty charts (convenience sample) were extracted in duplicate to ensure consistency between reviewers. Complete blood counts (CBC) were performed on the Sysmex XN-3000 analyzer. EPO testing was performed by enzyme linked immunosorbent assay (Bio-Techne, Minneapolis, MN, USA, R&D Systems). JAK2 molecular testing was performed by an allele specific polymerase chain reaction (AS-PCR) followed by allele separation and detection using the 4200 TapeStation System (Agilent Technologies, Santa Clara, CA, USA) with a lower limit of detection of 1–5% mutant allele burden.

2.3. Outcomes

Our primary objective was to determine the sensitivity (Sn), specificity (Sp), positive likelihood ratio (+LR), and negative likelihood ratio (−LR) of serum EPO, JAKPOT, and a combination of EPO and JAKPOT to identify patients with JAK2 mutant erythrocytosis. Our secondary objective was to identify adjusted and unadjusted clinical and laboratory predictors of JAK2 mutant erythrocytosis.

2.4. Statistical Analysis

We summarized patient characteristics descriptively with categorical variables expressed as percentages and quantitative variables as means with standard deviations. We calculated the Sn, Sp, +LR, and −LR of low EPO, JAKPOT, and EPO-JAKPOT for the diagnosis of JAK2 mutant erythrocytosis from 2 × 2 confusion matrices and high EPO for JAK2 mutant negative erythrocytosis (Supplementary Materials, Tables S1–S4). Low EPO was defined as <3.8 mU/mL, which is the lower limit of the reference interval for our institution’s assay. Sn was defined as True Positives [TP]/(TP + False Negatives [FN]), and Sp was defined as True Negatives [TN]/(TN + False Positives [FP]). A total of 95% confidence intervals (CI) were calculated using the Wilson efficient score method [10].

We identified univariate clinical and laboratory predictors of JAK2 mutant erythrocytosis with Mann–Whitney U-tests, two-tailed Welch t-tests, Chi-squared tests, and Fisher exact tests where appropriate and performed a multivariate logistic regression analysis with a subset of the predictors. Predictors in the multivariate analysis were chosen based on the completeness of the data and on theoretical grounds rather than by automated variable selection. Multicollinearity was assessed by calculation of variance inflation factors (Supplementary Materials, Table S5). Twenty-nine (14%) missing ferritin values in the logistic regression analysis were imputed using multiple imputation by chained equations [11]. A sensitivity analysis was performed showing results of the logistic regression analysis without missing value imputation (Supplementary Materials, Table S6). p < 0.05 was used to denote statistical significance for all analyses. p values were corrected using the Holm–Bonferroni method for multiple comparisons. We used Python (v3.11.9) for statistical analyses with packages numpy (v1.26.4), pandas (v2.2.2), statsmodels (v0.14.2), pingouin (v0.5.3), scipy (v1.13.1), and scikit-learn (v1.4.2).

3. Results

We identified 609 patients with JAK2 testing performed between 31 December 2017 and 31 December 2022. We included 213 patients after excluding patients who were anemic, had no serum EPO measured, had incomplete records, or were known to have undergone therapeutic phlebotomy or who donated blood within three months of testing (Figure 1). The mean age of included patients was 57.7 years, and 163 patients were male (69%) (Table 1). The mean hemoglobin was 174 g/L, mean serum EPO level was 10.7 mU/mL, and 40 (19%) patients had a JAK2 mutation (39 [18%] JAK2 V617F, and 1 patient with a JAK2 exon 12 mutation). Ten patients (5%) were on sodium–glucose cotransporter 2 (SGLT2) inhibitors; all were JAK2-negative (Table 1). No patients were receiving cytoreductive therapy before testing.

3.1. Performance of Low EPO or JAKPOT to Diagnose JAK2-Positive Erythrocytosis

Low EPO (<3.8 mU/mL) had a Sn of 0.77 (95% CI, 0.62–0.87), Sp of 0.98 (95% CI, 0.94–0.99), −LR of 0.23 (95% CI, 0.13–0.41), and +LR of 33 (95% CI, 12.5–89.6) for the diagnosis of JAK2 mutant erythrocytosis (Table 2, and Supplementary Materials Table S1). Elevated EPO (>16.9 mU/mL) had a Sn of 0.16 (95% CI, 0.11–0.22), Sp of 1.0 (95% CI, 0.91–1.0), −LR of 0.84, and an infinite +LR for the diagnosis of JAK2 negative (secondary) erythrocytosis (Table S4). A positive JAKPOT score (Figure 2) had a Sn of 0.88 (95% CI, 0.73–0.94), Sp of 0.65 (95% CI, 0.57–0.72), −LR of 0.19 (95% CI, 0.08–0.44), and +LR 2.5 (95% CI, 2.0–3.2) for the diagnosis of JAK2 mutant erythrocytosis (Table 2, and Supplementary Materials Table S2).

3.2. Performance of EPO-JAKPOT to Diagnose JAK2-Positive Erythrocytosis

Patients with a low EPO level (<3.8 U/mL) and/or positive JAKPOT score were classified as EPO-JAKPOT positive (Figure 2). Adding EPO to the JAKPOT score improved sensitivity for diagnosing JAK2 mutant erythrocytosis without sacrificing specificity. A positive EPO-JAKPOT score had a Sn of 0.95 (95% CI, 0.83–0.98), Sp of 0.66 (95% CI, 0.58–0.72), −LR of 0.07 (95% CI, 0.02–0.30), and +LR of 2.7 (95% CI, 2.2–3.4) to diagnose JAK2 mutant erythrocytosis (Table 2, and Supplemental Materials Table S3).

3.3. Univariate Predictors of JAK2-Positive Erythrocytosis

Patients with JAK2-positive erythrocytosis had a lower mean EPO than patients with JAK2-negative erythrocytosis (4.4 mU/mL vs. 12.3 mU/mL; p < 0.01) (Table 1). Older age, higher platelet count, higher hematocrit, higher absolute RBC count, and lower ferritin were also associated with JAK2-positive erythrocytosis (p < 0.01 for all comparisons) (Table 1).

3.4. Adjusted (Multivariate) Predictors of JAK2-Positive Erythrocytosis

Older age (p < 0.01), lower EPO (<3.8 mU/mL) (p < 0.01), and higher platelet count (p < 0.01) were independently associated with JAK2 mutant erythrocytosis after adjusting for potential confounders (Table 3). Looking specifically at components of the JAKPOT score, only the platelet count (p < 0.01) was associated with JAK2 mutant erythrocytosis after adjustment (Table 3).

4. Discussion

In our cohort, the JAKPOT score was less sensitive for identifying patients with JAK2-positive erythrocytosis compared to the JAKPOT derivation study (Sn 0.88; 95% CI 0.74–0.95 vs. 0.95–1.0) [8]. We hypothesized that combining JAKPOT with low EPO into a unified score (EPO-JAKPOT) would improve diagnostic performance compared to using either JAKPOT or EPO alone. We found that EPO-JAKPOT had high sensitivity (0.95; 95% CI 0.83–0.98) and a low −LR (0.07; 95% CI 0.02–0.30) for JAK2 mutant erythrocytosis, meaning that primary erythrocytosis could potentially be ruled out in an EPO-JAKPOT negative patient.

We found that low EPO (<3.8 mU/mL) was highly specific (0.97; 95% CI, 0.94–0.99) but insensitive (0.76; 95% CI 0.61–0.86) for diagnosing JAK2 mutant erythrocytosis. Likewise, high EPO (>16.9 mU/mL) had high specificity (1.0; 95% CI 0.89–1.0) but low sensitivity (0.16; 95% CI, 0.11–0.21) for JAK2-negative (secondary) erythrocytosis. In other words, an EPO level outside of the assay’s reference range accurately classified patients as JAK2 mutant or JAK2 wild type, but most EPO levels in our cohort (70%) were in the normal range and, therefore, unhelpful. These findings are consistent with other retrospective studies, which report that EPO levels below 2.9 or 3.7 mU/mL are specific (Sp 0.94–0.98) but relatively insensitive (Sn 0.64–0.68) for diagnosing PV [12,13]. In one retrospective cohort, a sensitivity of 0.96 for JAK2 mutant erythrocytosis was achieved only after increasing the EPO cut-off to 7.15 mU/mL [6]. Likewise, elevated EPO (>15.1 mU/mL) is known to have high specificity for a diagnosis of secondary erythrocytosis (Sp 0.98) but is insensitive (Sn 0.47) [12].

Older age, low EPO, and higher platelet count predicted JAK2-positivity in our cohort after adjusting for confounders. Additional univariate predictors included higher hematocrit (p < 0.01), and lower ferritin (p < 0.01). Low ferritin also approached statistical significance in our multivariate analysis (Table 3). These findings are overall consistent with other studies evaluating clinical and laboratory characteristics of patients with PV [8,13,14] and, when present, may prompt clinicians to consider a diagnosis of primary erythrocytosis.

Ten patients (5%), all of whom were JAK2-negative, were on SGLT2-inhibitors. The clinical significance and optimal management of SGLT-2 inhibitor-induced erythrocytosis remains uncertain [15,16,17].

We speculate that a higher prevalence of JAK2 positivity (19% compared with 12%) [8] and our exclusion of patients known to have recently undergone therapeutic phlebotomy or blood donation may help explain why the sensitivity of JAKPOT differed in our cohort compared to the derivation study. The two cohorts were similar in terms of severity of erythrocytosis (mean hemoglobin 175 g/L vs. 177 g/L), mean age (57 vs. 58 years) and proportion of male patients (69 vs. 72%) and were both assembled from tertiary care hospitals. Other unreported differences in patient characteristics, including, for example, kidney function, baseline ferritin, or risk factors for secondary erythrocytosis (e.g., proportion of smoking patients), may also have played a role.

The strengths of our study were our inclusion of all patients with EPO and JAK2 testing at our institution over a 5-year period and our ability to collect detailed laboratory and clinical information on these patients. Limitations include our relatively small sample size, leading to wide confidence intervals, selection bias resulting from patients investigated at a tertiary care hospital, collection of laboratory data at only a single point in time, and incomplete data for some clinical and laboratory parameters (e.g., ferritin and spleen size) due to the retrospective nature of data collection. JAK2 exon 12 mutation testing was ordered at the discretion of the consulting physician and may have led to some missed JAK2 mutant cases. Similar to the JAKPOT score, our outcome of interest was detecting JAK2 mutant (primary) erythrocytosis rather than identifying patients with a formal WHO diagnosis of polycythemia vera. Weaknesses of our multivariate analysis include low event rate and multicollinearity between a subset of the predictors, but findings from this analysis are consistent with other cohorts. One drawback of EPO-JAKPOT, compared to the original JAKPOT score, is the need to measure serum EPO. However, EPO testing is recommended for the workup of undifferentiated erythrocytosis [18], and low EPO remains a minor criterion in the WHO 2018 criteria for the diagnosis of polycythemia vera.

Had our study cohort been screened with EPO-JAKPOT, we would have ruled out JAK2-positive erythrocytosis in 130 patients (55%) without molecular testing at the cost of only 2 missed cases (1%). The prevalence of JAK2 mutant erythrocytosis in primary care is unknown, but it is likely less than 1% compared to 19% in our cohort [19]. We expect that EPO-JAKPOT would have even better negative predictive value in primary care and other settings with a lower prevalence of JAK2 mutant erythrocytosis than what was observed in our study [20]. In summary, the EPO-JAKPOT score shows potential for excluding JAK2 mutant erythrocytosis in select patients without molecular testing. However, additional prospective validation is required before it can be adopted into routine clinical practice.