mRNA COVID-19 Vaccine Effectiveness Against Severe Outcomes Among Adults Hospitalized with COVID-19 from May 2021 to January 2023

TL;DR

This study shows that mRNA vaccines were effective in preventing severe outcomes from COVID-19 in hospitalized adults, especially during the pre-Delta and Delta periods.

Contribution

The study provides real-world evidence of mRNA vaccine effectiveness against severe outcomes during different SARS-CoV-2 variant periods.

Findings

mRNA vaccines showed 85.5% effectiveness against hospitalization during the pre-Delta/Delta era.

Effectiveness dropped to 38.2% during the Omicron period but was partially restored with a booster dose.

Vaccine effectiveness was highest within six months of vaccination and for ICU admissions.

Abstract

Background/Objectives: COVID-19 mRNA vaccines protect against hospitalization, but less is known about real-world vaccine effectiveness (VE) against other severe outcomes. Methods: We enrolled adults hospitalized with acute respiratory illness at two hospitals in Atlanta, Georgia, USA from May 2021 to January 2023. Participants were eligible if they had standard-of-care COVID-19 testing or provided an upper respiratory swab for analysis. Vaccination status was confirmed through the state registry. mRNA COVID-19 VE among those with severe outcomes was determined using a test-negative case–control design with stepwise logistic regression adjusting for confounding variables. Results: Of 1973 participants eligible for analysis, 886 (44.9%) were unvaccinated, 641 (32.5%) received a primary series, and 446 (22.6%) received a primary series plus ≥ 1 booster. A total of 734 (37.2%) were positive for COVID-19. During the pre-Delta/Delta (2 May 2021–19 December 2021) vs. Omicron (20 December 2021–31 January 2023) eras, adjusted COVID-19 mRNA VE of a primary series compared to no vaccination was 85.5% (95% CI: 77.0%, 90.8%) vs. 38.2% (95% CI: 11.5%, 56.8%) overall, 90.0% (95% CI: 82.6%, 94.2%) vs. 54.4% (95% CI: 9.0%, 77.1%) among those with radiographic pneumonia, and 94.4% (95% CI: 80.5%, 98.4%) vs. 62.5% (95% CI: 19.0%, 82.7%) among those admitted to the ICU. VE against severe outcomes was highest within the 6 months following vaccination and during the pre-Delta/Delta era. A booster dose partially restored VE against Omicron-associated hospitalization and pneumonia. Conclusions: COVID-19 mRNA vaccines were effective at preventing hospitalization and other severe outcomes in adults during periods of pre-Delta/Delta and Omicron variant circulation.

Figures

Full text

1. Introduction

COVID-19 has been associated with significant morbidity and mortality since the onset of the pandemic in March 2020. As of May 2024, there were over 100 million COVID-19 cases and over 1.2 million total deaths associated with COVID-19 in the United States (US) [1]. Certain populations, including those who are immunocompromised or have other underlying comorbidities, are at a higher risk of severe COVID-19 [2,3,4]. COVID-19 vaccines have played an important role in protecting against infection, severe disease, and death due to COVID-19 [5,6].

Two mRNA-based vaccines for COVID-19, BNT162b2 (Pfizer Inc., New York, NY USA) and mRNA-1273 (Moderna, Inc. Cambridge, MA, USA), were made available initially through an Emergency Use Authorization (EUA) in December 2020, with full approval granted in August 2021 and January 2022, respectively [7,8]. Vaccine effectiveness (VE) against symptomatic COVID-19 was high for both vaccines in Phase III trials, at 95.0% for BNT162b2 and 94.1% for mRNA-1273 vaccine in SARS-CoV-2 naïve populations [9,10,11]. However, real-world VE decreased from the original estimates due to factors such as the emergence of SARS-CoV-2 variants, including Delta and Omicron with its subvariants [12,13,14]. Vaccine recommendations evolved over time to include the addition of booster doses in alignment with predominant circulating variants [15,16], and these recommendations continue to evolve. It is therefore critical to provide real-world evidence to inform decision-making about vaccine policies and recommendations. While it is known that COVID-19 vaccines offer protection against mortality, with COVID-19 vaccines preventing an estimated 20 million deaths worldwide during the first year of vaccine administration alone, additional information about VE against severe disease outcomes, including prolonged hospitalization and ICU admission, is important in the setting of waning immunity and emergence of new SARS-CoV-2 variants [17,18].

As the majority of the US population has now been exposed to COVID-19, hybrid immunity has become more common: the combination of infection- and vaccine-induced immunity. By the end of 2022, an estimated 96.7% (95% CI: 93.9%, 98.3%) of individuals of the US aged 16+ were seropositive for SARS-CoV-2, with 77.5% (95% CI: 75.1%, 79.8%) having evidence of infection-induced seropositivity [19]. Like immunity to COVID-19 from vaccination alone, hybrid immunity wanes over time from the last exposure to SARS-CoV-2, but prior research suggests that it may confer additional protection compared to prior infection alone and increased protection with increased doses of COVID-19 vaccines [20,21,22]. Nevertheless, the capacity of hybrid immunity to confer protection against severe COVID-19 outcomes is incompletely understood, in part due to the differential effects of infection timing and variant on disease outcomes [23,24].

Additionally, the effectiveness of COVID-19 vaccines against severe outcomes among racial and ethnic minority groups is incompletely understood. This is important because COVID-19 has caused disproportionate burden of disease among some groups during different waves of the pandemic. For example, a systemic review and meta-analysis of 4.3 million patients from 68 studies found that members of racial and ethnic minority groups, including African Americans, Hispanic, and Asian Americans had higher rates of COVID-19 positivity and disease severity than White populations [25]. Furthermore, people of Black race and Mixed races were found to have significantly lower rates of COVID-19 vaccination compared to White populations [26]. As COVID-19 vaccination has been found to reduce disparities in outcomes among certain groups [27], understanding VE among diverse populations could help inform vaccine recommendations.

This study aimed to estimate the effectiveness of a primary series and booster dose of mRNA COVID-19 vaccines and secondarily of hybrid immunity using a test-negative, case–control design among hospitalized participants and those experiencing other severe outcomes. The study was performed at two hospitals in Atlanta, Georgia, where there is a diverse patient population with a predominance of African American patients. This manuscript expands upon data previously published demonstrating the effectiveness of the BNT162b2 vaccine against hospitalization [28].

2. Materials and Methods

2.1. Participant Enrollment and Identification

For this test-negative, case–control study, participants were eligible for enrollment if they were ≥18 years of age, were hospitalized at one of two metropolitan Atlanta hospitals (Emory University Hospital, EUH; Emory University Hospital Midtown, EUHM) for acute respiratory infection (ARI) between May 2021 and January 2023 and had previously provided a nasopharyngeal (NP) or nasal swab as part of standard-of-care (SOC) testing or were willing to provide an NP/nasal swab specimen at the time of study enrollment and comply with all requested data collection. ARI was defined as having either ARI symptoms, including nasal congestion, rhinorrhea, sore throat, hoarseness, new or increased-from-baseline cough, sputum production, dyspnea, or wheezing, or having an admission diagnosis suggestive of ARI, including pneumonia, upper respiratory infection (URI), bronchitis, cough, asthma exacerbation, viral respiratory illness, and respiratory distress. Potential participants were excluded if they had received SARS-CoV-2 directed antiviral treatment within the preceding 30 days or if they had received COVID-19 monoclonal antibody therapy or COVID-19 convalescent serum within the preceding 90 days of NP or nasal swab collection.

Participant demographic, epidemiological, and clinical data were collected from a patient interview, when possible, and the electronic medical record (EMR). Data was stored in a REDCap database. Vaccination data were collected from the EMR and verified through the Georgia Immunization Registry (GRITS). Discrepancies between the EMR and GRITS were adjudicated by the study principal investigator and data manager. Participants were considered ‘Unvaccinated’ if they had not received any prior COVID-19 vaccination. Participants who had received the initially recommended primary series vaccination regimen were considered to have a primary series: 2 doses of any mRNA vaccine for immunocompetent patients, and 3 doses of any mRNA vaccine for immunocompromised patients. Patients were classified in the boosted group if they had received at least one additional dose of any mRNA vaccine after meeting the primary series eligibility. There was no minimum time interval requirement between doses for participants. Participants were considered immunocompromised if they had a history of cancer, HIV/AIDS, solid organ transplant, hematopoietic stem cell transplant, splenectomy, chronic steroid use, or rheumatologic conditions. A diagnosis of pneumonia was defined using radiographic reports from the medical chart. “Definite” pneumonia required clear notation in the radiologist’s report. If the radiologist report did not state pneumonia to be the clear and only diagnosis but included consolidation, pleural effusion (not “trace” or “possible”), or alveolar/interstitial infiltrates, or if the final diagnosis included pneumonia/infection plus one other etiology then participants were categorized as having “probable” pneumonia. These were captured in the database as a combined variable of definite/probable pneumonia. Participants were asked to report prior history of SARS-CoV-2 infection during the patient interview. Hybrid immunity was defined as receiving mRNA vaccination as described above and having history of SARS-CoV-2 infection noted in the medical chart. Self-reported prior infection data was used to inform time since prior infection estimates if the medical records did not provide adequate details. Participants who had received partial primary COVID-19 vaccination schedules, those who had received a non-mRNA COVID-19 vaccine, and those who had received COVID-19 vaccination within 14 days of hospital admission were excluded from this analysis.

The study was reviewed and approved by the Institutional Review Board at Emory University, and written informed consent was obtained from eligible participants or their legally authorized representatives as applicable.

2.2. COVID-19 Testing

Participants were tested for respiratory pathogens by a standard-of-care (SOC) molecular test, typically either GeneXpert (Cepheid, Sunnyvale, CA, USA) or BioFire^®^ (Salt Lake City, UT, USA) respiratory viral panel (RVP) tests, or study-specific testing [29]. Study-specific testing used combined NP/oropharyngeal (OP) swabs with a BioFire^®^ RVP.

2.3. Statistical Analyses

COVID-19 cases were defined as participants who tested positive for COVID-19 by any source, whether SOC molecular testing or the BioFire^®^ RVP, within 14 days prior to enrollment. Controls were defined as those negative for COVID-19 by all testing sources and methods. Potentially eligible participants whose COVID-19 swab was collected greater than 72 h after admission were excluded from the study to avoid potential nosocomial infections. Although the incubation period for nosocomial infections varies by SARS-CoV-2 variant, it typically ranges from 3 to 6 days [30]. Sociodemographic and clinical characteristics were compared using bivariate analyses. T-tests, Chi-squared, or Fisher exact tests were used as appropriate. A p-value ≤ 0.05 was considered statistically significant when comparing cases vs. controls. Bonferroni correction was applied when comparing unvaccinated vs. primary series and vs. booster dose to reduce the risk of type I errors in the setting of multiple bivariate comparisons; these results were considered statistically significant with p-values ≤ 0.025. Absolute VE compared to no vaccination was estimated with a test-negative, case–control design using the formula VE = (1 − odds ratio (OR)) × 100%, where OR was defined as the odds of COVID-19 vaccination (primary series or boosted) among COVID-19-positive cases divided by the odds of vaccination among COVID-19-negative controls. As a secondary analysis, to assess the impact of hybrid immunity, the exposure status of cases and controls were stratified by any self-reported history of infection by comparing hybrid immunity to unvaccinated individuals without a history of prior infection. Stepwise multivariate logistic regression was performed to find the adjusted COVID-19 mRNA VE, with a model inclusion criterion of 0.05. ORs were adjusted for age, enrollment quarter (i.e., calendar time by 3-month intervals), sex, race/ethnicity, immunocompromised status, and enrollment site. Severe outcomes included hospitalization, duration of hospitalization ≥ 4 days, definite/probable pneumonia, ICU admission, mechanical ventilation, and death. Four days was chosen as the cutoff duration for severe hospitalization, as it was the median duration of hospitalization among the cohort. VE estimates for the pre-Delta/Delta variant-predominant and Omicron variant-predominant eras were calculated by restricting the analyses to participants admitted during those periods (2 May 2021 to 19 December 2021, and 20 December 2021 to 31 January 2023, respectively). All analyses were performed using SAS v9.4 (SAS Institute Inc, Cary, NC, USA).

3. Results

3.1. Study Population

Of 5141 eligible patients, 2763 were enrolled (53.8%) (Figure 1). The demographic characteristics of enrolled and non-enrolled patients are shown in Supplementary Table S1. 790 participants (28.6%) were excluded from this analysis due to either receipt of a partial series of mRNA vaccine (n = 420), not having a vaccine record (n = 185), receipt of a COVID-19 vaccine within 14 days of admission (n = 67), receipt of a non-mRNA COVID-19 vaccine (n = 59), or no recorded COVID-19 testing (n = 59) (Figure 1). Of the 1973 remaining participants, 886 (44.9%) were unvaccinated, 641 (32.5%) received the primary series, and 446 (22.6%) received ≥ 1 booster dose. The distribution of homologous and heterologous vaccine schedules is shown in the Supplementary Table S2. The median time from the most recent dose of vaccine to hospital admission was 259 days [Interquartile Range (IQR): 143, 385] for participants who received a primary series and 153 days [IQR: 75, 274] for participants who received ≥ 1 booster dose. 1239 participants (62.8%) tested negative for SARS-CoV-2 (‘Controls’), and 734 (37.2%) tested positive (‘Cases’) (Figure 1). The distribution of other pathogens detected is shown in the Supplementary Table S3.

COVID-19 cases were younger than controls (median age 56 vs. 61 years, p < 0.0001), were admitted earlier in the study period than controls (p < 0.0001) and were less commonly admitted during the Omicron-predominant period (51.9% vs. 80.9%, p < 0.0001) (Table 1). Cases more commonly had children <18 years residing at home (30.1% vs. 19.6%, p < 0.0001), were more commonly employed (39.6% vs. 21.0%, p < 0.0001), were more commonly adherent to social distancing and masking guidance during the time period, had more commonly traveled in the month prior to enrollment (17.0% vs. 8.8%, p < 0.0001), and less commonly reported current or prior smoking (p < 0.0001) (Table 2). Cases also had a higher body mass index (BMI) (median 29.8 vs. 28.1, p = 0.014) but otherwise were generally healthier than controls and less commonly had a medical history of chronic respiratory disease, blood disorders, an immunocompromising condition, cardiac disease, or stroke. COVID-19 cases were more commonly pregnant than controls (3.1% vs. 1.7%, p = 0.036) (Table 3). In terms of disease severity, cases less commonly required CPAP/BIPAP (8.6% vs. 11.5%, p = 0.038) but more commonly had definite/probable radiographic pneumonia (59.7% vs. 47.2%, p < 0.0001) and duration of hospitalization ≥ 4 days (54.9% vs. 50.2%, p = 0.043) (Table 3).

Participants vaccinated with a primary series and boosted participants were older than unvaccinated participants (median age 63 and 69 vs. 52, p < 0.0001), were less commonly Black race (58.7% and 47.5% vs. 72.7%, p < 0.0001), were admitted later in the study period and were more commonly admitted during the Omicron-predominant period (76.4% and 98.0% vs. 51.6%, p < 0.0001) (Table 1). Participants with a primary series and boosted participants less commonly had children <18 residing in the home (16.1% and 13.7% vs. 33.9%, p < 0.0001), were less commonly employed (25.1% and 17.3% vs. 35.3%, p < 0.0001), more commonly reported past, non-current smoking history (33.1% and 36.1% vs. 26.2%, p < 0.0001), and had lower basic and instrumental activities of daily living scores (p < 0.0001) (Table 2). Primary series and booster vaccinated participants had lower BMIs (median 28.2 and 28.2 vs. 29.3, p = 0.033 and p = 0.010), more commonly had history of chronic respiratory disease, immunosuppressive conditions, cardiac disease, chronic kidney disease, dementia, diabetes, stroke, and home oxygen use. In terms of severity, they less often developed definite/probable radiographic pneumonia (52.6% and 38.1% vs. 58.2%, p = 0.028 and p < 0.0001) and less commonly had duration of hospitalization ≥ 4 days (49.5% and 49.1% vs. 55.2%, p = 0.027 and 0.036), although these results were not all statistically significant (Table 3).

3.2. Vaccine Effectiveness Analyses

After controlling for age, enrollment quarter (i.e., calendar time by 3-month intervals), sex, race/ethnicity, immunocompromised status, and enrollment site, VE among hospitalized participants admitted during the pre-Delta/Delta variant predominant period was 85.5% (95% CI: 77.0%, 90.8%) for an mRNA COVID-19 vaccine primary series and 96.4% (95% CI: 68.9%, 99.6%) for ≥1 booster (Table 4). For participants admitted to the hospital during the Omicron-predominant period, adjusted VE among hospitalized participants was 38.2% (95% CI: 11.5%, 56.8%) for a primary series and 48.5% (95% CI: 20.8%, 66.5%) for those with ≥1 booster dose.

In terms of other severe outcomes, for participants admitted during the pre-Delta/Delta era, the adjusted VE of an mRNA COVID-19 vaccine primary series was 90.0% (95% CI: 82.6%, 94.2%) against definite/probable radiographic pneumonia, 83.6% (95% CI: 68.4%, 91.5%) against duration of hospitalization ≥ 4 days, and 94.4% (95% CI: 80.5%, 94.4%) against ICU admission. We were unable to calculate VE of ≥1 booster dose during the pre-Delta/Delta era due to the small number of participants who received a booster dose during this period. For participants admitted during the Omicron era, the adjusted VE of an mRNA COVID-19 vaccine primary series was 42.9% (95% CI: 2.3%, 66.7%) against definite/probable radiographic pneumonia, 49.7% (95% CI: 16.3%, 69.8%) against duration of hospitalization ≥ 4 days, and 62.5% (95% CI: 19.0%, 82.7%) against ICU admission. A booster dose partially restored VE against severe outcomes during the Omicron era. The adjusted VE of ≥1 booster dose of an mRNA COVID-19 vaccine was 54.4% (95% CI: 9.0%, 77.1%) against definite/probable radiographic pneumonia, 57.4% (95% CI: 20.6%, 77.2%) against duration of hospitalization ≥ 4 days, and 48.7% (95% CI: −40.9%, 81.4%) against ICU admission.

Overall, VE was generally higher during the pre-Delta/Delta era than the Omicron era (Table 4). When stratified by time since vaccination (<6 months vs. ≥6 months) (Table 4), VE estimates were generally higher within 6 months of vaccination. When stratified by age group (18–64 years and ≥65 years) (Supplementary Table S4), VE estimates were generally higher among older adults. VE estimates against more rare severe outcomes (i.e., mechanical ventilation and death) were not evaluable due to small number of participants with these outcomes in the study (Supplementary Table S5).

As a secondary analysis, we determined the effectiveness of hybrid immunity against the same severe outcomes and hypothesized that the additional antigen exposure from natural infection would be associated with higher levels of protection. For participants admitted during the pre-Delta/Delta era, the adjusted effectiveness of an mRNA COVID-19 vaccine primary series plus self-reported history of COVID-19 when compared to unvaccinated participants without history of COVID-19 was 92.8% (95% CI: 63.5%, 98.6%) against COVID-19 associated hospitalization and 92.7% (95% CI: 60.6%, 98.6%) against definite/probable radiographic pneumonia, while hybrid immunity effectiveness against duration of hospitalization ≥ 4 days was not statistically significant and against ICU admission was not evaluable due to sample size (Table 5). Effectiveness of hybrid immunity with ≥1 booster dose was not evaluable during the pre-Delta/Delta era (against all severe outcomes) or during the Omicron era (against ICU admission, mechanical ventilation, or death) due to the small number of participants with hybrid immunity and these outcomes. For participants admitted during the Omicron era, point estimates of adjusted VE of an mRNA COVID-19 vaccine primary series and ≥1 booster dose against COVID-19 associated hospitalization, radiographic pneumonia, and duration of hospitalization ≥ 4 days were positive, but not statistically significant.

VE power calculations are shown in Supplementary Tables S6–S8. Based on these, the study was underpowered to ascertain VE against some of the stratified outcomes, including VE of booster doses during the Pre-Delta/Delta era (due to insufficient booster vaccination coverage) and effectiveness of hybrid immunity during the Omicron era (due to lower numbers of cases and lower effectiveness against severe outcomes).

4. Discussion

mRNA COVID-19 vaccines were effective against hospitalization and severe outcomes among adults hospitalized for acute respiratory illness at two hospitals in Atlanta, Georgia with a predominantly African American population from May 2021 to January 2023. VE was greatest during the pre-Delta/Delta era and within 6 months of vaccination. The primary series alone and with ≥1 booster vaccine provided significant protection from definite/probable radiographic pneumonia, duration of hospitalization ≥ 4 days, and ICU admission. Because almost half (44.9%) of the study population was unvaccinated at enrollment, opportunity existed for increasing vaccine coverage to our diverse study population to reduce the risk of COVID-19 related hospitalization and severe outcomes during the study period.

Our pre-Delta/Delta estimates of mRNA COVID-19 VE among hospitalized participants were high and comparable to other studies during the early pandemic according to a systematic literature review [31]. A booster dose almost completely protected against hospitalization during the pre-Delta/Delta era, also consistent with prior studies [32]. Our VE estimates of the primary series and ≥1 booster dose decreased during the Omicron era, as has been previously described [32,33], and were likely attributable in part to immune evasion by this variant [34]. Many studies have found higher VE of a booster dose when compared with a primary series alone. However, a systematic review of the effectiveness of pre-Omicron COVID-19 vaccines against the Omicron variant in comparison to Delta found reduced VE against Omicron-associated hospitalization and severe outcomes that was only partially boostable and quickly waned [35]. Multiple factors may modify booster dose VE, including the prime and boosting strains, the time interval since the most recent booster dose, and the currently circulating variants and their potential to evade pre-existing immunity [36]. Immunologic imprinting is thought to play a key role in individual responses to novel variants, as the initial exposing strains direct subsequent responses to slightly different antigens encountered in the context of B cell memory [37]. In our study, VE of ≥1 booster dose was higher against most evaluable severe outcomes than the primary series alone within both the pre-Delta/Delta and Omicron eras; however, VE of ≥1 booster dose notably waned and was no longer statistically significant >6 months after vaccination during the Omicron era. While we were unable to calculate a statistically significant VE for COVID-19 associated death due to the small number of observed events in our study population, larger studies have found significant protection against mortality [32,38].

In terms of hybrid immunity, previous studies have found a higher, more durable effectiveness of hybrid immunity compared to COVID-19 vaccination alone [39]. Hybrid immunity has been found to increase mucosal immunity [40] and to broaden humoral [41] and cellular responses [42]. A post hoc cross-protocol analysis of Moderna, AstraZeneca, Janssen, and Novavax phase 3 COVID-19 vaccine clinical trials found that hybrid immunity provided near-complete protection against severe disease across platforms [21]. In our study, we found that hybrid immunity was effective among those who had received a primary series and had disease onset during the pre-Delta/Delta era; however, limitations in sample size during the Omicron era precluded our ability to draw firm conclusions about hybrid immunity in that era. Point estimates for effectiveness of hybrid immunity among those who had additionally received a booster dose and among those enrolled during the Omicron era were positive, but not statistically significant. This lack of observed significant effectiveness may have been attributable to limitations in sample size or to misclassification of subclinical infections due to recall bias. Another possibility is that prior infection had less impact among those who were infected with a different variant due to immune imprinting. The need to overcome immune imprinting in order to confer protection to emerging variants coinciding with waning pre-existing immunity underscores the need for revaccination [43], which remains the cornerstone to protect against severe COVID-19 and to mitigate future outbreaks.

In terms of public policy recommendations, the Advisory Committee for Immunization Practices (ACIP) to the Centers for Disease Control and Prevention (CDC) has historically made immunization recommendations based on a scientific evidence to recommendation (EtR) framework [44]. This framework has included the critical assessment of the public health importance of a disease, the risks and benefits of vaccination, the values and acceptability among the affected population, resource utilization, equity, feasibility, and finally, the balance of consequences. Real-world safety and effectiveness analyses have been iterative and ongoing. Thus, there has not been a VE “threshold” that alone has been clinically meaningful. Rather, VE in the context of the above domains and the public policy goals have guided vaccine recommendations. For COVID-19, a public policy goal has been to reduce hospitalizations and other severe disease manifestations. Pooled VE estimates of the 2023–2024 and bivalent formulations used to inform the 2024–2025 EtRs showed pooled VE of 44% (95% CI 34–52%) against COVID-19 associated hospitalizations in adolescents and adults and 23% (95% CI 8–36%) against COVID-19 associated deaths [45]. While we did not find statistically significant VE against mortality, our VE estimates against other severe outcomes during the Omicron era were generally similar to those of the pooled EtR analyses, albeit with waning after 6 months. Thus our findings align with and support the recommendation for COVID-19 primary series and booster in the era of Omicron circulation to prevent severe disease.

This study has limitations. A large proportion of potentially eligible individuals did not enroll, and more were excluded based on vaccine history, which may have led to selection bias that differed by group. For example, if non-enrolled patients were sicker and less commonly vaccinated, this may have led to an underestimation of VE against severe outcomes. Potentially eligible participants whose COVID-19 swab was collected greater than 72 h after admission were excluded from the study to avoid potential nosocomial infections, but this may have also led to a selection bias. There were some baseline differences between cases and controls, such as adherence to social distancing behaviors, that could have impacted COVID-19 exposure and VE estimates.

We adjusted for confounding using step-wise multivariable logistic regression; however, this type of model is prone to overfitting and inflated type I errors. Some VE estimates, particularly for outcomes like death or mechanical ventilation, were unstable due to insufficient sample size. Further, the broad definition of definite/probable pneumonia may have led to misclassification of incidental radiographic findings unrelated to clinical pneumonia. There was likely overlap in the severe outcomes participants experienced, and thus the VE estimates for each outcome were not independent from each other. The study period spanned two distinct eras with different vaccination rates, variant circulation, and baseline immunity. While we adjusted VE estimates for calendar time by quarter, there may have been residual impact of temporal confounding. This analysis was completed prior to the emergence of newer Omicron sub-variants, such as the JN.1 lineage, and future research is therefore needed to assess VE against these and other variants with updated antigen-specific monovalent vaccines. In addition, all diagnoses of prior COVID-19 may not have been accurately captured due to recall bias in the interview, if a participant interview was not possible, if prior infections were mild or asymptomatic, or lack of prior diagnostic testing. These limitations may have impacted our ability to assess hybrid immunity, and future research is needed to better understand the impact of hybrid immunity in preventing severe outcomes. The sample size was insufficient to directly compare vaccine types (mRNA-1273 vs. BNT162b2) or booster antigen specificities (prototype vs. bivalent BA.4/5), calculate VE for different strains of SARS-CoV-2 beyond VE estimates during periods of variant predominance, or calculate VE amongst participants with certain severe outcomes, although important differences may exist. In addition, a test-negative, case–control design has a few inherent limitations, such as the impact of vaccination or COVID-19 status misclassification errors on VE estimates and the assumption that COVID-19 vaccination does not affect the incidence of COVID-19-like illness [46,47,48]. Lastly, as this study occurred at only two sites within the same hospital system, the results may not be generalizable to all healthcare settings.

5. Conclusions

In conclusion, using prospective active surveillance and a test-negative, case–control design, we found that mRNA COVID-19 vaccination provided significant protection among participants experiencing severe outcomes, including COVID-19 associated hospitalization, pneumonia, hospitalization duration ≥ 4 days, and ICU admission among a predominantly African American study population in Atlanta, Georgia. These results provide further evidence of the benefits vaccines have had in the prevention of severe COVID-19 through the pre-Delta/Delta and Omicron eras. Future research should continue to investigate the real-world effectiveness of variant-specific COVID-19 vaccines in the setting of newly emerging SARS-CoV-2 variants.