A Reanalysis of the FDA’s Benefit–Risk Assessment of Moderna’s mRNA-1273 COVID Vaccine Based on a Model Incorporating Benefits Derived from Prior COVID Infection

TL;DR

This study reevaluates the FDA's benefit-risk analysis of Moderna's mRNA-1273 vaccine for young adult males, finding that vaccine risks may outweigh benefits unless Omicron infection rates are very high.

Contribution

The study introduces a refined benefit-risk model that incorporates benefits from prior infection and provides more accurate risk stratification for vaccination.

Findings

Vaccine risks outweigh benefits for 18–25-year-old males unless Omicron infection rates are implausibly high.

Vaccination of 18–25-year-old males resulted in 8% to 52% more hospitalizations for VAM/P than prevented by the vaccine.

Model inputs and outputs were validated by data available after the FDA's original assessment.

Abstract

Background: The U.S. Food and Drug Administration (FDA) conducted a benefit–risk assessment for Moderna’s COVID vaccine mRNA-1273 prior to its full approval, announced 31 January 2022. The FDA’s assessment focused on males 18–64 years old because its risk analysis was limited to vaccine-attributable myocarditis/pericarditis (VAM/P), given the excess risk among males. The FDA’s analysis concluded that vaccine benefits outweighed risks, even for 18–25-year-old males (those at highest VAM/P risk). We reanalyze the FDA’s benefit–risk assessment using information available through the third week of January 2022 and focusing on 18–25-year-old males. Methods: We develop a benefit–risk model, extending the FDA’s, that can stratify benefits and risks of vaccination by prior-infection and comorbidity status. We use the FDA’s framework but apply our model to account for benefits derived from prior COVID infection, while also accounting for finer age stratification in COVID-hospitalization rates, incidental hospitalizations (those of patients who test positive for COVID but receive treatment for something else), more realistic projections of Omicron-infection rates, and more accurate VAM/P rates. Results: With hospitalizations as the principal endpoint of the analysis (those prevented by vaccination vs. those caused by VAM/P), our model finds vaccine risks outweighed benefits for 18–25-year-old males, except in scenarios projecting implausibly high Omicron-infection prevalence. Our assessment suggests that mRNA-1273 vaccination of 18–25-year-old males generated between 8% and 52% more hospitalizations for VAM/P compared to COVID hospitalizations prevented (over a five-month period of vaccine protection assumed by the FDA). The preceding assessment uses model inputs based on data available at the time of the FDA’s mRNA-1273 assessment. Moreover, these inputs as well as model outputs are validated by subsequently available data. Conclusions: The outcome of a vaccine benefit–risk assessment may be dramatically impacted by accounting for the benefits derived from prior infection by the vaccine-targeted disease. To increase public confidence in vaccines and thereby reduce vaccine hesitancy, public-health agencies should employ benefit–risk models capable of supporting stratification of vaccination recommendations not only based on age and sex but also on prior-infection and comorbidity status.

Figures

Tables

Table 3. FDA scenarios projecting benefit–risk outcomes per million 18–25-year-old males vaccinated with two primary series doses of Moderna RNA-1273 *.

Scenario	Benefits (Outcomes Prevented)				VAM/P Risks
	Cases	Hospitalizations	ICUVisits	Deaths	Cases	Hospitalizations	ICUVisits	Deaths
BLA-Memo	76,362	1755	421	4	148	128	47	0
1	82,484	4766	1144	51	128	110	0	0
2	26,705	2088	501	48	128	110	0	0
3	3903	635	152	7	128	110	0	0
4	219,958	5957	1430	63	128	110	0	0
5	82,484	4766	1144	51	68	58	0	0
6	82,484	4766	1144	51	247	207	0	0

Table 4. Summary of input variables *.

Variables for U and V	Range of Values	Scenario A Values
Ir: Baseline-infection rate—fraction of an unvaccinated, infection-naïve test population of 18–25-year-old males expected to contract an infection over the 5-month evaluation period. Infections need not accumulate at a constant rate	0≤Ir≤1	1
Hr: Baseline infection-hospitalization rate for infection-naïve unvaccinated males 18–25	0≤Hr≤0.01	0.0014
Fpi: Fraction of a test population of 1 million unvaccinated males 18–25 having had a COVID-19 infection before the start of the 5-month evaluation period	0≤Fpi≤1	0.62
Epi: Effectiveness of prior-infection protection against COVID-19 infection	0≤Epi≤1	0.45
Ev: Effectiveness of vaccination against COVID-19 infection	0≤Ev≤1	0.30
Eh: Effectiveness of hybrid protection against COVID-19 infection	0≤Eh≤1	0.57
HRRpi: Hospitalization-risk reduction for those reinfected and unvaccinated	0≤HRRpi≤1	0.79
HRRv: Hospitalization-risk reduction for those fully vaccinated and experiencing their first COVID-19 infection	0≤HRRv≤1	0.67
HRRh: Hospitalization-risk reduction for those reinfected and fully vaccinated	0≤HRRh≤1	0.88
HVAM/P: Projected number of VAM/P hospitalizations occurring in the course of attaining a test population of 1 million fully mRNA-1273 vaccinated 18–25-year-old males	HVAM/P≈268	268

Table 7. Hospitalization-risk ratios for a typical male 18–25 without prior-infection protection: risk of hospitalization after choosing mRNA-1273 vaccination (VAM/P or severe COVID-breakthrough infection *) divided by risk of COVID hospitalization after choosing not to vaccinate.

Probabilityof Infection Ir	Probability ofCOVID-19 HospitalizationWithout mRNA-1273 Vaccination	Probability ofCOVID-19 HospitalizationAfter mRNA-1273 Vaccination	Hospitalization Risk RatioVaccinated/Unvaccinated, withVAM/P Risk 268 in 1 Million
1	1400 in 1 million	323.4 in 1 million	0.42≈268+323.41400
0.5	700 in 1 million	161.7 in 1 million	0.61≈268+161.7700
0.25	350 in 1 million	80.85 in 1 million	1.00≈268+80.85350

Full text

1. Introduction

Because vaccines can be given to large populations of healthy persons and can be introduced as mandatory, robust benefit–risk analyses must be conducted to ensure that vaccines have highly favorable benefit–risk profiles [1].

The Food and Drug Administration (FDA) issued emergency-use authorizations for mRNA COVID-19 vaccines from Pfizer (BNT162b2) and Moderna (mRNA-1273) in December 2020 [2]. Once the mRNA COVID-19 vaccines were given to large populations, it became evident that they could cause myocarditis or pericarditis (hereafter myo/pericarditis), inflammation of the heart or of the membrane surrounding the heart, respectively, with studies indicating the highest risk among males under 40 years old [3,4,5,6,7,8,9]. Possible mechanisms by which mRNA COVID vaccination causes myocarditis are explored in [10,11].

The FDA published two benefit–risk analyses of primary series mRNA vaccination, the first for BNT162b2 [12] and the second for mRNA-1273 [13]. To assess vaccine benefits, the FDA estimated vaccine-preventable COVID-19 cases, hospitalizations, intensive-care unit (ICU) admissions, and deaths [12,13]. To assess vaccine risks, the FDA estimated vaccine-attributable myo/pericarditis (VAM/P) cases, hospitalizations, ICU admissions, and deaths [12,13]. The FDA concluded that the benefits of the mRNA vaccines outweighed VAM/P risk even for males in its highest-risk age groups: 16–17 years for BNT162b2 [12] and 18–25 years for mRNA-1273 [13]. To simplify our exposition, we focus mainly on the FDA’s analysis for Moderna’s mRNA-1273—there are significant differences in modeling assumptions in the FDA’s Pfizer and Moderna assessments. Also, VAM/P risk is substantially greater for mRNA-1273 than for BNT162b2 [3,4,5,6,7,8,9].

Our benefit–risk modeling differs from the FDA’s primarily by its accounting for the protection conferred by prior infection, and, at the time of the FDA’s Moderna analysis (mid-January 2022), likely over 70% of 18–25-year-olds in the U.S. had been infected by COVID-19—see Supplement S1. On the risk side, we use data available at the time of the FDA’s mRNA-1273 assessment, including that from [12,14,15,16,17], suggesting VAM/P risk for mRNA-1273 exceeded FDA estimates; see Section 1.4 below.

Here, our principal aim is to demonstrate how to incorporate the protection derived from prior infection into a population-level quantitative benefit–risk model for a two-dose vaccination series. Our model has hospitalizations as its endpoint, comparing the hospitalizations-prevented benefit of vaccination to the hospitalization risk attributable to vaccination. We illustrate the utility of our model by applying it to extend and improve the FDA’s mRNA-1273 assessment relative to its hospitalizations endpoint, which can be viewed as the most important of the agency’s endpoints owing to the public-health concern that during waves of the COVID pandemic hospitals might be so overwhelmed that they would not be able to deliver high quality care to their patients.

In addition to prior-infection protection, our modeling accounts for incidental hospitalizations, age-specific infection-hospitalization rates, and a more accurate assessment of VAM/P risk. We apply our model using data available to the FDA at the time of its benefit–risk analysis for mRNA-1273, assumed to be the 3rd week of January 2022 (see Section 1.2). We present evidence, using real-world data, that our model’s estimates of the hospitalizations-prevented benefit of mRNA-1273 vaccination are significantly more accurate than those the FDA obtained in [13]. We also analyze benefits and risks of vaccination for specific subpopulations such as males 18–25 having prior-infection protection and no comorbidities. Our model can be adapted to produce a benefit–risk assessment for any vaccine designed for deployment on a population-wide scale, stratified not only based on age and sex but also on prior-infection and comorbidity status.

We emphasize that the application of our model to reanalyze the FDA’s benefit–risk assessment for mRNA-1273 is for demonstration purposes and does not constitute a comprehensive re-evaluation, which would address all of the FDA’s benefit–risk endpoints (cases, hospitalizations, ICU admissions, deaths).

1.1. Framework of the FDA’s mRNA-1273 Benefit–Risk Assessment

In [13], the FDA describes its benefit–risk modeling of Moderna’s mRNA-1273 for males ages 18 to 64 years. Outcomes are provided via six scenarios reflecting uncertainties in three major model inputs: COVID-19 incidence, vaccine effectiveness against cases ( $[eqn]$ ) as well as hospitalization ( $[eqn]$ ), and vaccine-attributable myo/pericarditis (VAM/P) rate. At the time of the FDA’s mRNA-1273 analysis, the CDC was recommending a 5-month interval between primary-series completion and a booster dose. Consequently, the FDA evaluated benefits and risks over a 5-month period, which we take to be 1 January 2022–31 May 2022. The FDA assumed that all vaccine effectiveness estimates remained constant over this 5-month evaluation period. The FDA’s major modeling assumptions are summarized in Table 1 below.

Using inputs from the first row of Table 1 and incidence data from [13] [Table 3], the FDA concluded for Scenario 1, its “most likely scenario”, that vaccinating one million males ages 18–25 years with two doses would prevent, over 5 months, 82,484 COVID-19 cases, 4766 COVID-19 hospitalizations, 1144 ICU admissions, and 51 deaths due to COVID-19, while risking 128 VAM/P cases, 110 VAM/P hospitalizations, 0 ICU admissions and 0 deaths from VAM/P [13] [Table 2]. Observe that the FDA does include a high VAM/P-risk scenario (6) as well as lower-benefit scenarios (2 and 3).

1.2. The Time of the FDA’s mRNA-1273 Analysis

Although the FDA does not define in [13] the time of its benefit–risk analysis for mRNA-1273, a natural choice is the 3rd week of January 2022, which concluded 10 days before the FDA’s announcement of the full approval of mRNA-1273 on 31 January 2022 [18]. This choice is consistent with the FDA’s time-of-analysis definition for its BNT162b2 assessment: the 2nd week of August 2021 [12] [Section 2.1], which concluded nine days before the FDA’s announcement of the full approval of BNT162b2 on 23 August 2021 [19]. Consequently, our reanalysis of the FDA’s benefit–risk assessment [13] is based on model inputs derived from information available before 22 January 2022. Subsequent information validating our input choices will often be provided but only via footnotes. Starting in Section 3.2, we discontinue basing our modeling on information available only before 22 January 2022; in Section 3.2, our intent is to show how our model can be modified to incorporate a comorbidity (obesity) as a risk modifier.

1.3. The FDA’s mRNA-1273 Benefit Assessment

The FDA’s computation in [13] of the cases-prevented and hospitalizations-prevented benefits of vaccination is straightforward. For each scenario, CDC data is used to anticipate over the 5-month evaluation period a certain number of COVID cases C and hospitalizations H per million unvaccinated males, and the corresponding evaluation-period benefits per 1 million vaccinations are estimated to be

$$[eqn]$$

where $[eqn]$ and $[eqn]$ are, respectively, the effectiveness of full mRNA-1273 vaccination against cases and against hospitalization; the FDA assumed that for Omicron $[eqn]$ and $[eqn]$ , as noted in Table 1 above.

For Scenario 1, its most likely scenario, and two others, the FDA assumed that peak 2021 Omicron-case rates (late December 2021) would be sustained for 5 months, resulting in an exceptionally high projection of COVID-19 cases among the unvaccinated; e.g., C = 274,947 for 18–25-year-old males. Similarly, for these scenarios, the FDA assumed that the CDC’s reported late-December 2021 Omicron-hospitalization rate for unvaccinated 18–45-year-old males would be sustained for 5 months and assumed that the same rate would apply to the younger 18–25 bracket, arriving at $[eqn]$ hospitalizations. Thus, for these scenarios, the FDA predicted that full mRNA-1273 vaccination of 1 million 18–25-year-old males would prevent $[eqn]$ COVID cases and $[eqn]$ COVID hospitalizations over the 5-month evaluation period [13] [Table 2].

How our benefit assessment differs from the FDA’s:

•* We account for the protection conferred by prior infection.* At the time of the FDA’s mRNA-1273 assessment, there was substantial evidence that prior infection would provide significant protection against COVID infection and hospitalization. In a study published 5 January 2021, Dan et al. found “About 95% of subjects retained immune memory at ∼6 months after [COVID] infection” [20]. A meta-analysis of studies published December 2020 through August 2021 demonstrated that protection from prior COVID infection “is, at least, equivalent to the protection afforded by complete vaccination of COVID-naïve populations” [21]. The CDC acknowledged in a 29 October 2021 science brief that “The immunity provided by vaccine and prior infection are both high” [22], while another CDC study found that during the Delta wave, unvaccinated persons with prior COVID infection had significantly greater protection against infection and hospitalization than did fully vaccinated, COVID-naïve persons [23]. Moreover, in December 2021, the FDA acknowledged, “The omicron variant has significantly more mutations than previous SARS-CoV-2 variants, particularly in its S-gene, the gene that encodes the virus’s spike protein” [24], suggesting that prior infection, entailing exposure to all SARS-CoV-2 proteins (e.g., the nucleoprotein), would likely yield greater protection against Omicron than would mRNA-1273 vaccination, which entails exposure only to the spike protein of the ancestral strain of SARS-CoV-2. Among the findings of a study of SARS-CoV-2 specific B and T cell memory is “CD8+ T cell responses preferentially target the nucleoprotein, highlighting the potential importance of including the nucleoprotein in future vaccines” [25]1.

Further evidence of the need to account for prior-infection protection includes (i) a CDC estimate that 54.9% of 18–49 year-olds had been infected by COVID-19 by 30 September 2021 [31], (ii) a South African study [32] released 14 December 2021 suggesting overall effectiveness of prior-infection protection against Omicron to be about 47% [Supplement S1, Section S4], and (iii) data from England [33] released 22 December 2021 suggesting that prior infection among the unvaccinated provides significant protection from Omicron hospitalization [Supplement S1, Section S5].

•* We account for incidental COVID-19 hospitalizations.* At the time of its mRNA-1273 assessment, incidental-hospitalization rates were well known to be substantial. On 9 January 2022, the director of the CDC stated, “Up to 40 percent of the patients who are coming in with COVID-19 are coming in … with something else” [34]2. On 10 January 2022, Ontario began publishing incidental rates, with rates for 8–15 January exceeding 45% [36]. There is evidence that incidental rates are especially high for the young. For example, a major study [37] found, “As of May 2021, the cumulative rate for hospitalization associated with SARS-CoV-2 infection was 35.9 per 10,000 children” and “When limiting the analysis to include only patients with COVID-19 who required therapeutic interventions, the hospitalization rate decreased 5.5-fold to 6.5 per 10,000 children”, which suggests an incidental rate of over 80% “for children” (persons 0–17-years-old). See Supplement S1 for additional incidental-hospitalization data.

•* We account for finer age stratification in COVID-hospitalization rates.* The FDA assumed that hospital admission rates were uniform for males of ages 18–45, contradicting CDC models indicating that hospitalization rates for those 30–49 are two times the rates for 18–29-year-olds [38]. In contrast, we employ an Omicron infection-hospitalization rate estimate for precisely the group of interest—males, 18–25 years old. Our estimate, based on a model developed in [39], is consistent with COVID-hospitalization data for the U.S. military [Supplement S1, Section S2].

•* We use a CDC case-to-infections multiplier to inform our projections of Omicron-infection prevalence.* The FDA assumed in its most likely scenario (as well as Scenarios 5 and 6) that case and hospitalization rates would remain constant at peak 2021 Omicron levels for 5 months. Throughout the pandemic, the number of COVID infections has always been significantly larger than the corresponding number of reported cases. For the period February 2020 through September 2021, the CDC estimated the number of infections to be four times the corresponding number of reported cases [31]. Moreover, the case-to-infection multiplier likely increased (e.g., owing to increased home testing3) during the Omicron-dominant period in the U.S., which started mid-December 2021. If we conservatively assume that the CDC’s 4:1 infection:case ratio holds over the evaluation period, we find the FDA’s evaluation-period case-count projection of 274,947 per 1 million unvaccinated 18–25-year-old males (Scenarios 1, 5, 6) corresponds to 4 · 274,947 = 1,099,788 infections, well over 1 million. Rounding down, we deem the FDA’s case-total assumption to be equivalent to the assumption that among one million unvaccinated males, there would be, over 5 months, 1,000,000 infections. Having well over one million infections in a group of 1 million 18–25-year-olds over 5 months is implausible. Multiple infections over a 5-month period would be unlikely given the high protection from reinfection resulting from recovery from a recent infection, and any reinfections would be offset by individuals who did not become infected at all (e.g., owing to prior-infection protection, limited exposure to infectious persons, or, possibly, genetic factors [41])4. Thus, for 3 of the FDA’s 5 Omicron-based scenarios, including its most likely scenario, the FDA essentially assumed that every unvaccinated male 18–25 would be infected by COVID over the evaluation period.

In contrast, for our most likely scenario, we project COVID incidence over the evaluation period to be twice that of the 2nd COVID wave in the U.S. (1 October 2020–28 February 2021), arriving at an estimate that 45.6% of unvaccinated, COVID-naïve 18–25 year-old males in the U.S. would be infected by Omicron over the evaluation period—see Section S1 of Supplement S15.

1.4. The FDA’s mRNA-1273 Risk Assessment

The FDA relied on its Biologics Effectiveness and Safety (BEST) System for the VAM/P-case rates appearing in Table 1 above and used the CDC’s Vaccine Safety Datalink (VSD) system to estimate the percentage of cases requiring hospitalization [13] [Section 2.3.2].

VAM/P-case rates vary considerably depending on the case definition employed and the extent of stratification of risk, based on age, sex, vaccine manufacturer, and dose, with most cases occurring postdose 2 [9]. For its mRNA-1273 assessment, the FDA’s four BEST-System data partners reported, with wide confidence intervals, the following myo/pericarditis incidence rates for 18–25-year-old males during the 7-day interval following dose 2: approximately 35, 70, 146, and 256 cases per million 2nd doses [44] [p. 12]. The FDA’s BEST System uses health-claims data not validated by a complete review of patients’ medical charts [13] [Section 4] and has “inherent limitations, such as small sample sizes and imperfect sensitivity of ICD-10 codes to identify these rare outcomes” [12] [Section 4]. We rely on four sources of data to estimate the VAM/P rate for dose 2 among 18–25-year-old males: (i) Buchan et al. [14], who apply the Brighton Collaboration’s case definition, (ii) Patone et al. [15], who consider only hospitalized cases of myocarditis, (iii) Sharff et al. [16], who apply the CDC’s case definition [45], and (iv) the FDA (with VSD-based adjustments).

How our mRNA-1273 VAM/P-risk assessment differs from the FDA’s:

•* We include dose-one VAM/P events.* For its mRNA-1273 analysis [13], the FDA computed the evaluation-period benefits of two doses of mRNA-1273 without accounting for the VAM/P risk associated with dose 1, potentially biasing the FDA’s analysis in favor of vaccination. In contrast, for BNT162b2-related VAM/P, the FDA added excess myo/pericarditis cases postdose 1 and postdose 2 to approximate the population-level risk of completing 1 million two-dose vaccination series [12] [Section 2.3.1]. Hence, for BNT162b2, the FDA did account for 1st-dose VAM/P risk but did not assess the benefits of dose 1 alone during the interval starting with dose 1 and ending at the point of full vaccination, potentially biasing the analysis against vaccination. Because we, like the FDA, do not account for adverse events of vaccination other than VAM/P, and mRNA vaccination might have unknown long-term adverse health impacts, we believe that even in a benefit–risk analysis assessing benefits only postdose 2, the choice to include 1st-dose VAM/P risk is appropriate, being consistent with the first-do-no-harm principle, the FDA’s choice in its BNT162b2 assessment, and the FDA’s choice in the benefit–risk analysis of its BLA (Biologics License Application) Clinical Review Memorandum for mRNA-1273—see Section 1.5 below. Thus, we include the 1st-dose VAM/P risk for mRNA-1273 in our main analysis. See Section S2 of Supplement S2 for a sensitivity analysis showing that accounting for dose 1’s benefits does not substantially change our benefit–risk conclusions.

•* We incorporate evidence that VAM/P risk for mRNA-1273 was over twice that for BNT162b2.* In a “Review Memorandum Addendum” [46], dated 19 November 2021, the FDA describes evidence that VAM/P risk for mRNA-1273 is significantly greater than that for BNT162b2, including regulatory-agency reports from five countries, data from Ontario later analyzed by Buchan et al. [14], and data from the CDC: “The most recent data presented by the Center for Disease Control and Prevention (CDC) analyzed from the Vaccine Safety Datalink (VSD) indicates a higher rate of myocarditis with the Moderna COVID-19 Vaccine than the Pfizer-BioNTech COVID-19 Vaccine ….”

The “most recent data” from the VSD to which the FDA refers was presented at the CDC’s October 2021 ACIP meeting [17]. Adjusted-rate ratios for 18–39-year-old males from the table on page 27 of [17] suggest the 2nd-dose VAM/P rate for mRNA-1273 is 2.26 times that of BNT162b2. Data from other sources suggests higher rate ratios, e.g., a 2nd-dose rate ratio of 5.1 was reported by Buchan et al. for males 18–24 [14] [p. 5] and of 5.2 was reported by Husby et al. for males 12–39 [47] [paragraph after Figure 3]. A study by Patone et al. of myocarditis cases requiring hospitalization within a 28-day risk interval after a vaccine dose found among males under 40 a rate of 101 excess myocarditis hospitalizations per million 2nd doses of mRNA-1273 and 12 per million 1st doses, with corresponding BNT162b2 rates being 12 per million 2nd doses and 3 per million 1st doses [15] [p. 4].

Based on “preliminary findings” [46] [p. 2], having “large uncertainty” owing to “small numbers of observed events” [44] [p. 16], as well as omitting dose-1 cases, the FDA suggests that for 18–25-year-old males, VAM/P risk for mRNA-1273 is comparable to that for BNT162b2: 128 cases per million 2nd doses of mRNA-1273 [13] [Table 4] vs. 131 cases per million full vaccinations of BNT162b2 (among 18–24-year-old males) [12] [Table 3], with corresponding hospitalization projections of 110 per million 2nd doses of mRNA-1273 and 131 per million full BNT162b2 vaccinations. Throughout this paper, we assume that VAM/P rates for males 18–24 well approximate those for males 18–25.

Using the rate ratios and BNT162b2 VAM/P-risk assessment discussed above, we compute a VAM/P-hospitalization rate of 224 per million 2nd doses of mRNA-1273 for males 18–25 years old [Supplement S1, Section S6.1].

The FDA’s BEST System also provided evidence that VAM/P risk for mRNA-1273 might be over twice that for BNT162b2. An FDA briefing document for a meeting held 14 October 2021 indicates that one of the agency’s BEST data partners reported a postvaccination myo/pericarditis incidence rate of 283.7 (95% CI 145.2–573.5) cases per million full mRNA-1273 vaccinations (7 day risk window after each dose) [48] [Section 2.3.3]. Applying a background rate of four cases per million per week for 18–25-year-old males for each dose [Supplement S1, Section S6.3], we find the VAM/P-case rate corresponding to 283.7 is 275.7 per million full vaccinations. Assuming a case-hospitalization rate of 100%, as in FDA’s BNT162b assessment, we obtain 275.7 VAM/P hospitalizations per million full mRNA-1273 vaccinations vs. 131 per million full BNT162b vaccinations.

As we have indicated, we rely on four sources of data for our VAM/P-rate estimates for dose 2 of mRNA-1273: the FDA with VSD-based adjustments, as well as studies by Buchan et al. [14], Patone et al. [15], and Sharff et al. [16] (from, respectively, Ontario, England, and the U.S.). The conclusion of [16], which includes the following, is noteworthy6:

We identified additional valid cases of myopericarditis following an mRNA vaccination that would be missed by the VSD’s search algorithm …. The true incidence of myopericarditis is markedly higher than the incidence reported to US advisory committees.

In Table 2 below, we compare the FDA’s projected mRNA-1273 VAM/P case and hospitalization rates to those of the other sources discussed above. We exclude VAM/P-rate sources, such as [49], depending on strictly passive surveillance—see Section S6.8 of Supplement S1.

1.5. A Seventh Benefit Risk Scenario for mRNA-1273

The FDA’s BLA Clinical Review Memorandum [51] for mRNA-1273, completed 28 January 2022, describes a benefit–risk scenario not matching any scenario of the FDA’s published benefit–risk assessment of mRNA-1273 [13]. The projections of the BLA-Memo Scenario [51] [Section 4.7] are described in the initial row of data of Table 3 below and compared to the FDA’s scenarios of [13] [Table 2] in particular to Scenario 1, the most likely scenario. Differences include the BLA-Memo Scenario’s accounting for 1st-dose VAM/P risk and projecting 1 million full vaccinations would cause 47 ICU admissions (vs. 0 admissions)7.

Consistent with the BLA-Memo Scenario, the FDA’s 31 January 2022 press release [18] announcing the full approval of mRNA-1273 indicates some mRNA-1273 VAM/P cases require ICU care.

To support its estimates of VAM/P-hospitalization and ICU-admission risk, the FDA in [13] cites a source of VSD data [53] with a collection-cutoff point of 21 August 2021, over four months before completion of its mRNA-1273 assessment. The VSD data from [53] cited by the FDA [13] is scant, reporting only 21 postvaccination myo/pericarditis cases for those 18–29 [53] [p. 16]. A comparison of Buchan et al.’s myo/pericarditis data from Ontario [14] [Table 4, collected 1 June 2021–4 September 2021] to VSD data through 19 October 2021 [17] [pp. 8, 16] shows that, among males and females ages 18–39, the number of myo/pericarditis cases per million 2nd doses of mRNA-1273 detected by Public Health Ontario is 3.76 times the number detected by the CDC’s VSD system [Supplement S2, Appendix S3].

2. Methods

Recall that for its cases and hospitalizations endpoints, the FDA’s benefit–risk model has inputs C and H (the numbers of anticipated evaluation-period COVID cases and hospitalizations, respectively, in a test population of one million unvaccinated males) as well as $[eqn]$ and $[eqn]$ (the fractions of these events prevented by full vaccination). These model inputs are combined to project the cases-prevented and hospitalizations-prevented benefits of vaccinating the test population, and these benefits are compared to the model’s corresponding risk inputs, the numbers of VAM/P cases and hospitalizations projected for the 2nd mRNA-1273 dose.

Our model, which has hospitalizations as its endpoint, is algebraic (like the FDA’s) and thus is widely accessible. Our model extends the FDA’s by not only incorporating prior-infection protection but also by being essentially equivalent to the FDA’s model if prior infection is assumed to provide no protection. Because the FDA relied on CDC data to supply projected values for C and H, the FDA’s model does not explicitly link cases to hospitalizations and similarly does not explicitly link $[eqn]$ and $[eqn]$ . In contrast, our model does provide these links. For example, it links infections, a key model input replacing reported cases, to hospitalizations via another key input—the infection-hospitalization rate IHR. Additional input variables are described below.

Model inputs are combined to project the hospitalizations-prevented benefit of vaccinating a test population, and this projected benefit is compared to the model’s risk input, the projected number of VAM/P hospitalizations expected to occur in the course of achieving a fully vaccinated test population. In our modeling, ”infection” is any COVID infection, symptomatic or asymptomatic.

Test population. Our model is based on a hypothetical test population of 1 million males aged 18–25. We assume that the test population is representative, as of 1 January 2022 of all males in this age group in terms of: distribution of ages within the 18–25 range, comorbidities (e.g., obesity, cardiovascular disease, diabetes), and the percentage having had a prior COVID-19 infection. We project the number of hospitalizations for Omicron infection that will occur in this test population over the evaluation period 1 January 2022–31 May 2022 (a) assuming all members remain unvaccinated throughout the period, and (b) assuming all members of the population are vaccinated throughout the period (all fully vaccinated, 14 days past dose 2, on 1 January 2022). The difference—the projection of (a) minus that of (b)—is the hospitalizations-prevented benefit of vaccinating the test population. We call a test population consisting of infection-naïve unvaccinated persons a baseline test population.

2.1. Model Inputs

Infection rate $[eqn]$ . We base our modeling on $[eqn]$ , the fraction of a baseline test population of unvaccinated infection-naïve males 18–25 projected to become infected (symptomatic and asymptomatic) over the FDA’s 5-month evaluation period.

*Infection-hospitalization rate * $[eqn]$ . We translate infections to projected hospitalizations using an estimated infection-hospitalization rate $[eqn]$ for unvaccinated, infection-naïve males 18–25 years old. $[eqn]$ is the fraction of a population of unvaccinated individuals experiencing a primary COVID infection that will progress to severe disease requiring hospitalization.

*Prior-infection fraction $[eqn]$ *. This is the fraction of the test population having had a COVID-19 infection before the start of the evaluation period.

Vaccine effectiveness against infection $[eqn]$ . This is the fraction by which the baseline infection rate $[eqn]$ is reduced due to the protection afforded by vaccination; i.e., the infection rate for a fully vaccinated test population is $[eqn]$ .

*Prior-infection and hybrid effectiveness against infection, $[eqn]$ and * $[eqn]$ . These are the fractions by which the baseline infection rate $[eqn]$ is reduced due, respectively, to the protection afforded by prior infection, or to both vaccination and prior infection (hybrid protection).

*Hospitalization-risk reduction * $[eqn]$ , $[eqn]$ , *and * $[eqn]$ . These are the fractions by which the hospitalization rate $[eqn]$ is reduced for populations that are protected, respectively, by vaccination, prior infection, or both (hybrid protection).

*VAM/P hospitalizations * $[eqn]$ . Our final model input $[eqn]$ is the projected number of VAM/P hospitalizations expected to occur in attaining a test population of 1 million fully mRNA-1273 vaccinated 18–25-year-old males. To fully vaccinate 1 million individuals with an mRNA COVID vaccine, more than 2 million doses must be administered (assuming, e.g., that those who experience myo/pericarditis related to dose 1 do not take dose 2), and the adverse events associated with each of the over 2 million doses reflect the cost to the population of obtaining the benefits of 1 million full vaccinations.

2.2. Combining Model Inputs to Produce Outputs

Recall that we are considering a test population of 1 million males 18–25 over the evaluation period 1 January 2022–31 May 2022. Let U denote the expected number of Omicron hospitalizations occurring in this test population over the 5-month evaluation period assuming its members remain unvaccinated throughout the period. Let V denote the corresponding number of expected hospitalizations assuming the population is vaccinated throughout the period. The difference $[eqn]$ is the hospitalizations-prevented benefit of vaccination for the test population, and the outputs U and V can be written in terms of the model inputs defined in the preceding section (thus, like the FDA’s, our model is deterministic).

In our modeling, we consider only the possibility of hospitalizations associated with test-population members’ first infections during the evaluation period. We expect second infections over a 5-month period to be rare; moreover, infection-hospitalization risk will drop significantly after infection [21,22,23]. Our decision to analyze hospitalization risk associated with only first infections during the 5-month evaluation period allows us to assume, as did the FDA, that all effectiveness factors contributing to the analysis remain constant over the evaluation period. Each effectiveness value represents average effectiveness over the evaluation period; in this way, our modeling accounts for waning of vaccine and prior-infection protection.

Deriving U. To derive U, we will assume that the test population remains unvaccinated during the evaluation period. As a first step, suppose that the entire test population is infection naïve. Then, in this baseline population, we expect $[eqn]$ infections to develop over 5 months, resulting in $[eqn]$ hospitalizations, where $[eqn]$ 1 million.

Next, suppose a fraction $[eqn]$ of the unvaccinated test population has had a COVID-19 infection prior to the start of the evaluation period. We must account for the protection against infection and hospitalization afforded by prior infection. We accomplish this in two stages, first accounting for protection against infection, then for protection from progression to severe disease for infected individuals. The first stage depends on the model input $[eqn]$ , while the second stage uses $[eqn]$ .

So, with fraction $[eqn]$ of the test population previously infected, $[eqn]$ are infection-naïve and will respond like the baseline population described above. That is, we expect $[eqn]$ infections leading to $[eqn]$ hospitalizations from this group. Among the remaining $[eqn]$ members of the test population with prior infection, we expect $[eqn]$ infections resulting in $[eqn]$ hospitalizations. Thus, the total number of hospitalizations expected in the unvaccinated test population is

$$[eqn]$$

Deriving V. Next, we assume that every individual in the test population is fully vaccinated. Since all members are vaccinated, those with prior infection have hybrid protection. Once again, we reason in two stages, first accounting for the protection against infection, then for the protection against hospitalization for infected individuals. Once again, we assume a fraction $[eqn]$ of the test population has had a prior COVID-19 infection.

Then, in the fully vaccinated test population, the expected number of hospitalizations is $[eqn]$ for those without prior infection and $[eqn]$ for those with hybrid protection. So, the total number of hospitalizations in the population is

$$[eqn]$$

Hospitalizations prevented by vaccination. As noted above, the difference $[eqn]$ is the hospitalizations-prevented benefit of vaccination for the test population:

$$[eqn]$$

We remark that disregarding the protection afforded by prior infection, as the FDA did, is equivalent in our model to assuming that all persons in the test population are infection-naïve, i.e., that $[eqn]$ . With $[eqn]$ in Equation (3), its right-hand side is

$$[eqn]$$

which is essentially the FDA’s model $[eqn]$ for hospitalizations prevented. The quantity $[eqn]$ of (4) corresponds to the $[eqn]$ formula on page 10 of [54], which is the reference the FDA used in [13] for its $[eqn]$ estimate of 0.72. Thus, our model for hospitalizations-prevented $[eqn]$ represents a natural extension of the FDA’s model.

2.3. Estimating Values for Model Inputs

Like the FDA, we account for uncertainties in model inputs by adopting a scenario-based assessment. The FDA’s model-input assumptions for its six scenarios of [13] are summarized in Table 1 above. Observe that the first three scenarios differ only in their COVID incidence assumptions, with Scenario 1 representing the FDA’s most likely scenario. Our first three scenarios, Scenarios A–C, are reanalyses, respectively, of the FDA’s Scenarios 1–3 because our COVID incidence assumptions are derived from the FDA’s. Based on information available at the time of the FDA’s mRNA-1273 analysis, we consider the COVID incidence assumption of our Scenario D more likely to be accurate than those of Scenarios A–C; also, for Scenario D, our values for other inputs represent our best estimates. Thus, Scenario D is our most likely scenario. In Scenario E, our final scenario, we assume that prior infection and mRNA-1273 vaccination provide equivalent protection against Omicron, which, for reasons discussed Section 1.3, we believe to be the minimum reasonable assumption of prior infection’s protectiveness relative to vaccination’s protectiveness. Model inputs and outputs for our scenarios are described in the next section, while the methods we use to estimate input values are described below and summarized in Table 4. Our estimates are based on data available to the FDA before 22 January 2022.

*Estimating $[eqn]$ *. As we noted in Section 1.3, the FDA’s COVID-case-rate assumption for its most likely scenario, Scenario 1, corresponds to the assumption that essentially all unprotected (i.e., unvaccinated) 18–25-year-old males become infected over the 5-month evaluation period. Thus, in Scenario A, our reanalysis of Scenario 1, we too assume all unprotected persons in the test population become infected. However, in our modeling, the unprotected are the infection-naïve unvaccinated. Assuming all such persons become infected over 5 months is equivalent to setting $[eqn]$ , our Scenario-A value for $[eqn]$ . As discussed in Section S1 of Supplement S1, for our Scenarios B and C, values of $[eqn]$ are determined by the FDA’s COVID-incidence assumptions for its Scenarios 2 and 3, respectively, while for our Scenarios D and E, $[eqn]$ is derived from the assumption that over the evaluation period the increase in COVID-infection level among 18–25-year-old males is twice that of the general population during the second COVID wave in the U.S. (1 October 2020–28 February 2021).

*Estimating $[eqn]$ *. We estimate $[eqn]$ using an IHR model [55] developed via a seroprevalence-based meta-analysis anchored to the ancestral strain of SARS-CoV-2 and available as a preprint in October 2021 [39]. Applying the model, we obtain a “with COVID” IHR of 0.37% for the infection-naïve unvaccinated of ages 18–25 and reduce this rate assuming 40% of hospitalizations attributed to COVID are incidental (see Section 1.3), arriving at an IHR of $[eqn]$ (for infection-naïve unvaccinated 18–25-year-olds). The 0.22% rate for the general population of 18–25-year-olds in the U.S. (males and females) is consistent with U.S. military COVID-hospitalization data [56]. See Supplement S1 for further details concerning our derivation of the IHR value $[eqn]$ .

In its benefit–risk assessments [12,13] for, respectively, Pfizer’s and Moderna’s mRNA vaccines, the FDA relied on sex-specific CDC hospitalizations data. The case and hospitalization rates per 100,000 persons in Table 2 of [12] suggest that for the 18–24 age range, females are 2.53 times as likely as males to be hospitalized with COVID-19 and 1.21 times as likely to have a case of COVID-198. In Section S2 of Supplement S1, we use hospitalization and case data from Table 2 of [12] to estimate the IHR for unvaccinated, infection-naïve males 18–25 to be $[eqn]$ ; thus, we set $[eqn]$ . This value agrees with a natural IHR estimate derived from the FDA’s BLA-Memo scenario (discussed in Section 1.5). For the BLA-Memo Scenario, vaccination prevents 76,362 cases and 1755 hospitalizations by providing 30% effectiveness against cases and 72% effectiveness against hospitalizations. Thus, the scenario assumes that $[eqn]$ cases among the unvaccinated would lead to $[eqn]$ hospitalizations, yielding a case hospitalization rate of ≈0.958%. Applying the CDC case-to-infection multiplier of 4 and assuming a 40% incidental COVID-hospitalization rate, we arrive at the estimate $[eqn]$ ) ≈ 0.0014.

*Estimating $[eqn]$ *. For our Scenarios A–C, we use the value $[eqn]$ , which is the average of (a) the CDC’s estimate, mentioned earlier, that 54.9% of those 18–49 years old had been infected by COVID-19 before 1 October 2021 [31] with (b) an estimate derived in Supplement S1 that 69% of unvaccinated male 18–25-year-olds had been infected by 1 January 2022. For Scenarios D and E, we use $[eqn]$ . These values of $[eqn]$ may be underestimates because they rely on the CDC’s infected rate of 54.9% by 1 October 2021 for the general population of 18–49 year-olds, and we would expect a somewhat higher infected rate among the unvaccinated.

*Estimating * $[eqn]$ , $[eqn]$ . Recall that the FDA used $[eqn]$ and $[eqn]$ for all its Omicron-based scenarios [13] [Table 1]. For reasons discussed in Supplement S1, for all our scenarios, we use $[eqn]$ and $[eqn]$ . Hence, we are assuming a $[eqn]$ of $[eqn]$ , which exceeds the $[eqn]$ used by FDA.

Estimating $[eqn]$ and $[eqn]$ . Our values $[eqn]$ and $[eqn]$ are derived (see Supplement S1) from a December 2021 study [32], based on data from South Africa9.

Estimating $[eqn]$ and $[eqn]$ . In Supplement S1, we use data from [33] (based on Omicron cases in England for the period 1 December 2021–14 December 2021) to estimate $[eqn]$ and $[eqn]$ 10.

Estimating $[eqn]$ . In Section 1.4, we combined four assessments of VAM/P risk based on data available before 1 January 2022 to obtain an estimate of 250 VAM/P hospitalizations per million 2nd doses of mRNA-1273 given to 18–25-year-old males. In Section S6.7 of Supplement S1, we discuss VAM/P-hospitalization risk among 18–25-year-old males for dose 1 of mRNA-1273, justifying a choice of 18 VAM/P hospitalizations per million 1st doses, which is the 1st-dose risk the FDA uses in its BLA-Memo Scenario—compare the first two rows of Table 3 above. Thus, for all our scenarios, we set $[eqn]$ 11.

3. Results

We begin with a flowchart, Figure 1, illustrating the outcome of Scenario D, our most likely scenario whose inputs represent our best estimates based on information available before 22 January 2022.

In Table 5 below, we compare—for all our scenarios—the hospitalizations-prevented benefit of mRNA-1273 vaccination to the corresponding VAM/P-hospitalizations risk, with Scenarios A–C being reanalyses of the corresponding FDA Scenarios 1–3.

We illustrate our model’s application by carrying out Scenario-A computations. Using Scenario A assumptions from Table 5 and our definitions (1) and (2) of U and V, we obtain U = (1 − 0.62) · (1 m) · 1 · 0.0014 + 0.62 · (1 m) · 1 · (1 − 0.45) · 0.0014 · (1 − 0.79) ≈ 632, and V = (1 − 0.62) · (1 m) · 1 · (1 − 0.30) · 0.0014 · (1 − 0.67) + 0.62 · 1 · (1 − 0.57) · 0.0014 · (1 − 0.88) ≈ 168. Thus, for Scenario A, the hospitalizations-prevented benefit of vaccination is projected to be $[eqn]$ per million full mRNA-1273 vaccinations, roughly one-tenth of the FDA’s Scenario-1 estimate 4766 [13]. Although Scenario A has a benefit–risk ratio above 1, its assumption that $[eqn]$ makes this scenario highly unlikely12. For the more likely scenarios B, D, and E, our modeling suggests that mRNA-1273 vaccination of 18–25-year-old males generated between 16% and 62% more hospitalizations for VAM/P alone compared to COVID hospitalizations prevented. To be conservative, we adjust these estimates downward.

As discussed in Section 1.4, for our main analysis (whose results are presented in Table 5 above), we have included 1st dose VAM/P risk without accounting for 1st dose benefits (which is the choice the FDA made in its BNT162b2 assessment [12] and its BLA-Memo assessment of mRNA-1273 [51]). This may be viewed as a conservative choice, being consistent with the principle that vaccines should have “highly favorable benefit risk profiles” [1]. Nevertheless, as we indicated in Section 1.4, including 1st dose VAM/P risk while not including 1st dose benefits potentially biases an assessment against vaccination. Thus, we adjust our results to remove this bias by comparing the hospitalizations-prevented benefit of two doses to the VAM/P risk attributable to dose two only. Although this adjustment potentially biases our outcomes analysis in favor of vaccination, we, conservatively, characterize the outcomes of our more likely scenarios in the following way: comparing our estimated 2nd-dose VAM/P risk at 250 hospitalizations per million to the hospitalizations-prevented benefit of two doses—165 per million in Scenarios B, 181 per million in Scenario D, and 232 per million in Scenario E—we conclude that mRNA-1273 vaccination of 18–25-year-old males generated between $[eqn]$ and $[eqn]$ more hospitalizations for VAM/P compared to COVID hospitalizations prevented, with an increase of $[eqn]$ for Scenario D. In particular, relative to hospitalizations, risks exceed benefits in both Scenario D, our most likely scenario, and in Scenario E, a “best-case scenario” for vaccination incorporating the minimum reasonable assumption of prior-infection protection vs. vaccine protection.

An ideal benefit–risk assessment of a two-dose vaccination series would account for the benefits and risks of both doses. We present such an assessment in Section S2 of Supplement S2 and provide some commentary on it in Section 3.3 below, where we discuss sensitivity analyses exploring how modifying model inputs impacts the benefit–risk picture.

3.1. Subpopulation Analyses Based on Prior-Infection Status

There is a compelling case for conducting a benefit–risk assessment for the subpopulation of unvaccinated 18–25-year-old males having prior-infection protection because prior infection not only decreases the hospitalizations-prevented benefit of vaccination but also increases the risk of severe adverse events (e.g., those described in [60,61,62,63,64,65,66]) after vaccination [67,68], with [67] reporting prior infection is “associated with an increased risk of severe side effects leading to hospital care (1.56 [risk ratio] (95%CI: 1.14–2.12)).” Moreover, prior infection may increase the risk of VAM/P [15]—see Appendix S4 of Supplement S2 for details13.

For the subpopulation of 18–25-year-old unvaccinated males having a COVID infection prior to the evaluation period 1 January 2022–31 May 2022, we can use our model to compute the expected number of COVID hospitalizations in various scenarios simply by setting $[eqn]$ while leaving all other inputs unchanged. For example, with $[eqn]$ in Scenario A, our model predicts $[eqn]$ hospitalizations per million and $[eqn]$ , yielding a hospitalizations-prevented benefit of 89.46 per million vaccinated, far below VAM/P-hospitalization risk (≈268 per million vaccinated).

As we discuss in Appendix S3 of Supplement S1, the preceding paragraph’s analysis may be reinterpreted in terms of evaluation-period hospitalization risk for a typical unvaccinated 18–25-year-old male COVID-infected prior to the evaluation period. For such a male, the total hospitalization risk of the choice to vaccinate is essentially the sum of his probability of VAM/P hospitalization and his probability of having a breakthrough-case COVID hospitalization during the evaluation period. Our analysis is summarized in Table 6 and Table 7 below in terms of hospitalization-risk ratios, with ratios < 1 indicating the choice to vaccinate decreases risk and >1 indicating it increases risk.

According to Table 6, our modeling suggests that for a typical unvaccinated 18–25-year-old male with prior infection, the choice to vaccinate created a hospitalization risk over twice that associated with the choice not to vaccinate even if $[eqn]$ (evaluation-period infection is certain for an infection-naïve male 18–25). Because a “typical male” has an “average comorbidities level”, the information in Table 6 is difficult to apply. However, if our typical unvaccinated male of Table 6 also has no comorbidities, then his hospitalization risk ratios *should exceed those of Table 6*14, because risk-ratio denominators should decrease given no comorbidities and there is no evidence that being free of comorbidities reduces VAM/P risk [70]15.

For an 18–25-year-old male without prior COVID infection, hospitalization-risk ratios become favorable for vaccination:

3.2. Subpopulation Analyses Based on Prior Infection Status and BMI

At this point, we discontinue confining to footnotes discussions and citations of data unavailable at the time of the FDA’s benefit-risk analysis of mRNA-1273.

The FDA acknowledges, “The health condition of individuals may remarkably impact the B-R profile and its evaluation” [13] [Section 4]. Comorbidities such as obesity may triple COVID-hospitalization risk [72]. Our model can produce a benefit–risk analysis based on comorbidity status by appropriately adjusting its infection-hospitalization-rate input $[eqn]$ . We illustrate the process in Appendix S3 of Supplement S1, focusing on obesity because evidence suggests that Body Mass Index (BMI) is the single most potent modifier of COVID-19 risk for those under 60 years old [73,74]. We use an assessment of COVID-hospitalization risk based on BMI that finds risk is minimized when BMI $[eqn]$ and that for $[eqn]$ risk is multiplied by a factor of approximately 1.09 (1.08, 1.10) for each unit increase in BMI [75]. The results of our modeling appear in Table 8 below.

Table 6 and Table 7 of the preceding section establish the expectation that for a “typical male” 18–25 (having an “average comorbidities level”), prior-infection status determines whether vaccination is beneficial. Consistent with this expectation, Table 8 above indicates that for those having BMI in the intermediate range 25–34, prior-infection status does determine whether vaccination is beneficial. However, for those with high BMI (35–44) in a high-infection-risk scenario, Table 8 indicates that vaccination becomes beneficial even for those with prior-infection protection.

3.3. Sensitivity Analyses

Perhaps the most important of our sensitivity analyses assesses how the inclusion of 1st-dose benefits in our Scenarios B–E changes the corresponding benefit–risk ratios. This analysis, carried out in Section S2 of Supplement S2, yields the following benefit–risk ratios for our more likely scenarios: 0.68 for B, 0.73 for D, and 0.96 for E. Estimating inputs for 1st-dose benefit modeling was especially challenging given the extremely limited data on 1st-dose benefits for Omicron available before 22 January 2022. Our having lower confidence in input estimates for 1st-dose modeling contributed to our decision to present the modeling only as a sensitivity analysis.

Also, in Section S2 of Supplement S2, we provide a systematic sensitivity analysis related to Scenario D, our most likely scenario. For each input, we hold all others constant at their Scenario D values, determining the extent of possible variation in the chosen input before equipoise is reached. For example, if we fix our model input values, except for $[eqn]$ , at their Scenario D levels, then benefit–risk equipoise is reached when $[eqn]$ , a value nearly 50% greater than the Scenario D value of $[eqn]$ . We also point out that the risk-exceeds-benefit conclusion of Scenario D is not sensitive to the value of $[eqn]$ : even if $[eqn]$ , the benefit–risk ratio of this scenario is $[eqn]$ . The insensitivity of the risk-exceeds-benefit outcome of Scenario D to significant changes in inputs from their Scenario D values increases confidence in the outcome.

3.4. Model Validation

In Supplement S2, we test our model using Ontario COVID-hospitalization data from the evaluation period of 1 January 2022–31 May 2022, showing its projections of the hospitalizations-prevented benefit of vaccination are more accurate than those of the FDA [13]. We use Ontario data for many reasons; e.g., unlike the CDC, Public Health Ontario chose to publish incidental COVID-hospitalization rates (starting 8 January 2022) [36]; see [Supplement S2, Section S3] for other reasons.

We test our model in two ways: (i) showing that it fits Ontario COVID-hospitalization data with inputs at plausible levels for Ontario over the evaluation period [Supplement S2, Section S8], and (ii) using Ontario COVID-hospitalization data to predict corresponding data for the U.S. [Supplement S2, Section S7].

Based on COVID-hospitalization counts from Public Health Ontario as well as corresponding incidental-hospitalization rates, we estimate that vaccinating one million 18–25-year-old Ontarians prevented 152 COVID hospitalizations over the evaluation period—see Table S2.5 of Supplement S2. Making various adjustments, e.g., accounting for the better health of Ontarians vs. Americans, we obtain an upper-bound estimate that 214 evaluation-period hospitalizations were prevented by one million full COVID vaccinations of 18–25–year-old males in the U.S.—see Table S2.7 of Supplement S2 and the discussion that follows it. Observe that 214 is an upper bound for our projections of Scenarios B–D (see Table 5) and a little below our projection 232 of Scenario E, but all of the FDA’s projections, ranging from 635 to 4766 for Omicron-based scenarios, substantially exceed 214.

Ontario data also supports the definitiveness of the results of Table 6 above given the male has no comorbidities, suggesting that 151 in 1 million is an overestimate of his evaluation-period COVID hospitalization risk [Supplement S2, Section S2.3].

4. Discussion

By accounting for prior-infection protection and using evidence-based model inputs, we find in our Scenarios B–E of Table 5 that risks of mRNA-1273 vaccination outweighed benefits for the general population of 18–25-year-old males, assuming hospitalizations are the principal endpoint of analysis. In particular, for our scenarios predicting plausible infection-level increases over the evaluation period (Scenarios B, D, and E), our model suggests that vaccination of 18–25-year-old males generated between 16% and 62% more hospitalizations for vaccine-attributable myo/pericarditis (VAM/P) alone compared to COVID hospitalizations prevented, or, discounting 1st-dose VAM/P risk, between 8% and 52% more hospitalizations. Recalling that the FDA found, “The benefits of vaccination with the Moderna COVID-19 vaccine clearly outweigh the risks, even among the male adolescent population” [13] [Section 4], we have demonstrated that the conclusions of a benefit–risk assessment for vaccination may be dramatically impacted by accounting for the benefits conferred by prior infection. We have also demonstrated in Table 6, Table 7 and Table 8 how the benefit–risk picture for vaccination may vary based on an individual’s prior-infection and comorbidity status. Finally, we have demonstrated how a benefit–risk assessment for a two-dose vaccination series may acknowledge potential biases relating to the inclusion or exclusion of 1st dose benefits or risks.

Consistent with our model’s finding that for 18–25-year-old males in the U.S., risks of mRNA-1273 vaccination outweighed benefits relative to hospitalizations, we note that France suspended mRNA-1273 use on 15 October 2021, making it available again on 8 November 2021 but at half-dose and only for those over 30 years [76] [p. 4]. Also observe our model’s finding that vaccination of 18–25-year-old males generated between 8% and 52% more hospitalizations for vaccine-attributable myo/pericarditis (VAM/P) compared to COVID hospitalizations prevented is consistent with our model-validation work discussed in Section 3.4: recall that we estimate that at most 214 evaluation-period hospitalizations were prevented by one million mRNA-1273 vaccinations of 18–25-year-old males in the U.S. [Supplement 2, Table S2.7], and note that our corresponding VAM/P-hospitalization projection of 268 is a little over 25% higher than 214.

We emphasize that the cardiac risk of COVID infection is incorporated into our modeling because our estimates of COVID hospitalizations include those for COVID-related cardiac issues. We address here assertions such as “Importantly, the risk of myocarditis after any dose of COVID-19 vaccine, in any population, is substantially lower than the risk of myocarditis after SARS-CoV-2 infection”, which appeared, without supporting references, in an April 2024 editorial of the European Heart Journal [77]. Contrary to the editorial’s assertion, Patone et al. find that, “In men younger than 40 years old, the number of excess myocarditis events [leading to hospitalization] per million people was higher after a 2nd dose of mRNA-1273 than after a positive SARS-CoV-2 test (97 [95% CI, 91–99] versus 16 [95% CI, 12–18])” [6]. Karlstad et al. also find significantly higher risk of hospitalized myocarditis after dose 2: 183.9 per million 2nd mRNA-1273 doses [5] [Table 2] vs. 13.7 per million positive tests [5] [Table 7] among males 16–24. Moreover, these myocarditis-incidence rates expressed in cases per million positive tests overstate the risk of myocarditis after COVID infection expressed in cases per million COVID infections (because many infections are not confirmed by positive tests, but essentially all infections among those seeking care for myocarditis are confirmed by positive tests) [78]. Consistent with [78], the authors of [79] note, “The attributable risk estimates for COVID-19 [associated myocarditis] used laboratory confirmed cases as the denominator and will be affected by the proportion of all SARS-CoV-2 infections captured by testing, precluding a direct comparison with vaccine-associated attributable risks”.

4.1. The Strength of Prior-Infection Protection

As noted in Section 1.3, at the time of the FDA’s mRNA-1273 assessment, immunological expectations and studies such as [21,22,23,25,32,33] suggested that prior-infection protection would likely equal or exceed vaccine protection among the COVID-naïve. In our modeling, we have assumed that vaccination’s effectiveness against hospitalization is 77% in all scenarios, while that of prior infection is 88.5% except in Scenario E.

A systematic review [30] involving 21 vaccine-effectiveness studies from the period 3 December 2021–7 April 2022 reports,

In contrast to vaccine effectiveness against delta severe disease, the majority of vaccine effectiveness estimates for omicron severe disease were below 75%; for example, thirteen (81%) of sixteen vaccine effectiveness estimates within three months of vaccination with the primary series were below 75% (Figure). Moreover, 13 (42%) vaccine-specific estimates fell below 50% at some point in time after vaccination.

In contrast, the meta-analysis [27] estimates prior infection’s effectiveness against severe Omicron to be over 80% through 6 months after infection, while the meta-analysis [28] [Table S2] estimates its effectiveness to be over 88% through 45 weeks after infection, with the former estimate based on studies from 1 January 2022–1 June 2022 and the latter, on studies published by 31 September 2022. In mid-January of 2022, the minimum reasonable evidence-based assumption would have been that prior-infection protection would equal vaccine protection (as in our Scenario E).

4.2. Assessing VAM/P Risk for mRNA-1273

Contributing to our finding mRNA-1273 vaccination’s risks exceeded benefits for 18–25-year-old males relative to hospitalizations is our assessment that VAM/P risk for mRNA-1273 is substantially greater than that suggested by the FDA’s BEST System. As detailed in Section 1.4 and Section 2.3, our mRNA-1273 VAM/P-risk estimate of 268 hospitalizations per million full vaccinations is based on data, available to the FDA at the time of its mRNA-1273 assessment, from Ontario [14], England [15], and the United States [12,16,44,51]. In Appendix S2 of Supplement S1, we pool data, unavailable at the time of the FDA’s mRNA-1273 assessment, from Scandinavia [5], France [7], and British Columbia [8] to obtain a VAM/P-hospitalization rate between 240 and 277 per million full, homologous mRNA-1273 vaccinations among males 18–25. A possible explanation for the FDA’s underestimating VAM/P risk for mRNA-1273 is that its BEST System has “inherent limitations, such as small sample sizes and imperfect sensitivity of ICD-10 codes to identify these rare outcomes” [12] [Section 4].

Recall that there is evidence that the CDC’s VSD system also underascertained VAM/P risk for mRNA-1273. As noted in Section 1.5, during the period 1 June 2021–4 September 2021 of “enhanced passive surveillance” among males and females 18–39, Public Health Ontario’s myo/pericarditis detection rate per million 2nd doses of mRNA-1273 was 3.76 times that of the CDC’s VSD system based on its data through 9 October 2021 [Supplement S2, Appendix S3]. The study [4] offers at least a partial explanation for the VSD’s lower detection rate by revealing shortcomings in the VSD’s search algorithm.

4.3. Limitations

Our principal aim has been to illustrate how prior-infection protection, as well as comorbidity status, can be integrated into a deterministic benefit–risk model that extends, in a natural way, the one the FDA employed in its assessment of mRNA-1273 vaccination. We have developed and presented our model in accordance with BRIVAC (Benefit–RIsk models applied to VACcines) guidelines, in particular, its item 17 “Sensitivity/scenario analyses” [80]. We acknowledge that it is desirable to employ more sophisticated approaches to benefit–risk modeling and to accounting for uncertainty in such modeling; for instance, Monte Carlo simulation modeling with Beta/Poisson distributions for model inputs, guiding input choices for simulations.

In applying our model to extend and improve the FDA’s mRNA-1273 assessment [13] relative to its hospitalizations endpoint, we have assumed that young males are hospitalized for COVID at a lower rate than young females, as did the FDA in [12,13]. Data from [12,57] suggests this rate difference is substantial; however, the rates of [12,57] include incidental hospitalizations, and incidental rates for males may differ from those for females. We have assumed, as did the FDA, that vaccination and prior infection offer essentially equivalent protection for males and females. Also, we have based our assessment of the difference in hospitalization rates between young males and females on data in Table 2 of [12] representing mostly pre-Delta cases. Relative to Omicron, sex-based differences in hospitalization rates might have changed. However, the rate differences provided by Table 1 of [57], based on two years of data including that of the initial Omicron wave, are consistent with those of Table 2 of [12].

COVID-hospitalization data from Ontario [Supplement S2] and from Connecticut [Supplement S1, Appendix S1] suggests that our Omicron IHR value $[eqn]$ for 18–25-year-old males is a reasonable estimate for the Omicron IHR for all 18–25-year-olds (general population mix of males and females). Thus, even if we assume males and females in this age range are hospitalized for COVID at roughly equivalent rates, our IHR value of 0.14% for 18–25-year-old males appears to be a good estimate.

In Section S2 of Supplement S1, based on data available before 22 January 2022, we obtain an Omicron IHR estimate for 18–25 year-olds (general-population mix of males and females) of 0.22%, not 0.14%. One possible explanation is that we may have underestimated the incidental COVID-hospitalization rate for 18–25 year-olds. Our IHR estimate for 18–25 year-olds of 0.22% results from assuming an incidental Omicron-hospitalization rate of 40%, a rate suggested by [34,36] for hospitalizations with COVID among patients of all ages. However, as noted in Section 1.3 (and discussed in detail in Supplement S1), there is strong evidence that incidental COVID-hospitalization rates are higher among the young than among the elderly. In particular, COVID-hospitalization data discussed in Appendix S1 of Supplement S1 suggests an incidental rate of over 70% among 15–24-year-olds in Connecticut during calendar year 2020. Assuming an incidental rate of about 62.2% (instead of 40%) transforms our IHR estimate of 0.22% into the estimate 0.14%.

For a vaccine having high effectiveness against infection that does not quickly wane, vaccination can have an important benefit not captured by our model—providing an enduring reduction in the level of infection in the population that may allow some members to avoid infection entirely through the contribution of vaccination to herd immunity.

Our risk assessment, like the FDA’s, ignores known, rare vaccine-related adverse events other than myo/pericarditis that may require hospitalization. Finally, our risk assessment, like the FDA’s, ignores the possibility that COVID-19 vaccination may have long-term health effects; for instance, the FDA’s package-insert information for Moderna’s mRNA-1273 begins, “SPIKEVAX has not been evaluated for carcinogenic, mutagenic potential, or impairment of male fertility in animals” [81] [Section 13].

5. Conclusions

Our model illustrates the importance of nuanced vaccine recommendations based on careful benefit–risk modeling that takes into account health status and prior infection(s), two covariates that were not included in the FDA’s models [12,13]. For healthy young males with prior infection, our model suggests that risks outweighed benefits (relative to hospitalizations) based on data available prior to licensure of the mRNA-1273 product. However, for higher-risk young males with obesity, and especially those with no prior infection, vaccination was found to confer benefit.

Risks of infection and vaccination, such as long COVID and VAM/P, must be carefully weighed to offer informed consent. Current long-COVID prevalence estimates among adults with confirmed SARS-CoV-2 infection are most reliably estimated in the range of 7–15% when using standardized definitions and representative sampling [82,83], and when using a conservative definition consistent with other post-viral syndromes, the WHO finds 2.8% among those younger than 20 [84].

With respect to VAM/P, more recent formulations of Moderna’s COVID vaccine have a reduced mRNA dose and VAM/P-incidence rates have declined [85]. Long-term VAM/P prognosis remains somewhat unclear. A CDC follow-up at 90 days found that although the majority (81%) of VAM/P cases recovered, 54% had ongoing abnormal cardiac findings and 26% were on daily cardiac medication; only 68% were cleared for all physical activity [86]. Clinical guidelines for return-to-sport following myocarditis call for 3–6 months’ activity restriction [87,88]. In addition, there is the unknown prognosis for complications like late gadolinium enhancement (LGE) [89].

The question of boosting the general population, including healthy young males, is beyond the scope of this paper and requires tremendous nuance, a fact widely recognized by fall 2021 [90]. The FDA indicated in May 2025 [91] that it anticipates needing data from RCTs to evaluate clinical outcomes prior to authorizing COVID-19 vaccines for healthy individuals ages 6 months to 64 years, while immunogenicity standards will likely suffice for those with at least one chronic condition and those over age 65. The agency’s move thus operationalizes the risk-stratified approach supported by our deterministic modeling and will generate data that can be used in more sophisticated modeling.