Arms-Based Meta-Analysis of Microbiological Endpoints of 88 VAP Prevention Studies Using Antimicrobial Versus Non-Antimicrobial Strategies—Towards ‘VAP-Zero’?

TL;DR

This study compares antimicrobial and non-antimicrobial strategies for preventing ventilator-associated pneumonia and finds that neither approach achieves zero cases.

Contribution

The study uses arm-based meta-analysis to reveal that antimicrobial interventions do not lead to VAP-zero, a finding not evident from traditional contrast-based analysis.

Findings

Antimicrobial interventions did not achieve VAP-zero overall or for specific pathogens like Staphylococcus aureus.

Contrast-based and arm-based meta-analyses showed similar effect sizes for VAP prevention interventions.

Summary VAP incidence in antimicrobial groups was similar to non-antimicrobial groups for all pathogens analyzed.

Abstract

Background/Objectives: In traditional contrast-based meta-analyses of randomized concurrent controlled trials (RCCTs), topical antibiotic prophylaxis (TAP) appears more effective than either antiseptic-based or non-antimicrobial-based interventions for preventing ventilator-associated pneumonia (VAP). The objective here is to use arm-based methods to determine whether this effectiveness translates towards achieving VAP-zero, both overall and specifically for VAP in association with Staphylococcus aureus, Pseudomonas aeruginosa, and Acinetobacter species among the same RCCTs. Methods: Data were extracted from RCCTs sourced primarily from Cochrane reviews of VAP prevention interventions. Arms-based and contrast-based methods of meta-analyses of the VAP prevention effect size and the VAP incidence per 100 patients receiving mechanical ventilation were obtained using random effects methods. Results: The VAP prevention intervention effect sizes derived by contrast-based versus arms-based meta-analyses were similar for each of the three broad types of interventions. The overall VAP prevention effect of antibiotic-based interventions by contrast-based and arms-based methods were 0.39 (95% confidence interval 0.33 to 0.46; n = 28) versus 0.39 (95% confidence interval 0.32 to 0.47; n = 28), respectively. Surprisingly, the arms-based analysis revealed that the summary VAP incidence, both overall and for each of Staphylococcus aureus, Pseudomonas aeruginosa, and Acinetobacter species within antibiotic intervention groups, were similar to the respective summary incidences within intervention groups of non-antimicrobial RCCTs. Conclusions: VAP-zero, both overall and in association with specific microbial sub-types, has remained elusive using antimicrobial-based interventions. This inference was not evident from a contrast-based analysis.

Figures

SROC plots of summary effect size of non-antimicrobial (a; top), antiseptic (b; middle), and antibiotic (c; bottom) interventions in preventing overall VAP. Pneumonia incidence among control and intervention groups with symbol size proportional to group size (hollow circles). The diagonal dotted line is the line of equivalence and the curved green line is the summary SROC curve. Also shown are the summary point (solid red square), the hierarchical summary ROC curve (green) with 95% confidence limits (dotted yellow inner ellipse), and 95% prediction limits (dotted purple outer ellipse).

SROC plots of summary effect size of non-antimicrobial (a; top), antiseptic (b; middle), and antibiotic (c; bottom) interventions in preventing Pseudomonas VAP. Pneumonia incidence among control and intervention groups with symbol size proportional to group size (hollow circles). The diagonal dotted line is the line of equivalence and the curved green line is the summary SROC curve. Also shown are the summary point (solid red square), the hierarchical summary ROC curve (green) with 95% confidence limits (dotted yellow inner ellipse), and 95% prediction limits (dotted purple outer ellipse).

SROC plots of summary effect size (and associated 95% confidence and 95% prediction ellipses) of non-antimicrobial (a; top), antiseptic (b; middle), and antibiotic (c; bottom) interventions in preventing Staphylococcus aureus VAP. The diagonal dotted line is the line of equivalence and the curved green line is the summary SROC curve. Also shown are the summary point (solid red square), the hierarchical summary ROC curve (green) with 95% confidence limits (dotted yellow inner ellipse), and 95% prediction limits (dotted purple outer ellipse).

Full text

1. Introduction

There are numerous randomized concurrent controlled trials of interventions to prevent the occurrence of ventilator-associated pneumonia (VAP) [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88]. The interventions can be broadly classified into non-antimicrobial- and antimicrobial-based interventions. The latter include antiseptic-based decontamination and antibiotic-based decontamination interventions. In total, 88 randomized concurrent controlled trials present data for VAP isolates including Staphylococcus aureus, Pseudomonas aeruginosa, and Acinetobacter species (Table 1).

In preventing VAP, an aspirational goal for many intensive care units is to approach VAP-zero, an aspiration in line with the VAP-zero initiative [101]. On the other hand, a contrary view is that VAP is inevitable, especially when considering ICU patient populations with greater lengths of stay [102,103,104,105].

Antimicrobial-based methods appear more effective than non-antimicrobial-based methods in traditional contrast-based meta-analyses, but these methods are unable to indicate the propensity of the intervention groups of the various RCCTs to approach VAP-zero. Such an appraisal would require each control and intervention arm of the randomized concurrent controlled trials analyzed separately in an arms-based meta-analysis. Arms-based meta-analysis can most simply be undertaken using methods applied to the analysis of diagnostic tests such as Summary Receiver Operator Characteristic (SROC) plots [106,107,108].

VAP occurs in 5 to 40% of patients undergoing mechanical ventilation in intensive care units [109,110,111,112,113,114,115]. It would be expected that the effect of non-antimicrobial-based interventions would be more similar in their prevention effect versus each of Staphylococcus aureus, Pseudomonas aeruginosa, and Acinetobacter species than would be the case for antimicrobial-based interventions. However, the impact of the various non-antimicrobial-based versus antimicrobial-based interventions on the occurrence of VAP in association with these various types of isolates is not clear. Specifically, might ‘VAP-zero’ be more readily attainable for VAP in association with some isolates rather than others and with one broad category of intervention? A contrast-based analysis cannot address this question.

There are four objectives here: first, to compare the VAP prevention effect size estimates of various prevention interventions within the literature as generated using contrast-based versus arms-based methods of meta-analysis; second, to triangulate the estimates here with the previous effect size summaries for non-antimicrobial-, antiseptic- and antibiotic-based interventions within the literature; third, to then also compare the prevention effect size estimates for VAP in association with Staphylococcus aureus, Pseudomonas aeruginosa, and Acinetobacter species; finally, using the arms-based approach including a meta-regression, compare the different interventions towards their propensity to attain VAP-zero.

2. Materials and Methods

2.1. Study Selection and Decant of Groups

The literature search used here (Figure 1) is as described previously [116,117]. Cochrane reviews and other systematic reviews [118,119,120,121,122,123] were used as the primary source of studies, with additional studies being found by snowball sampling using the “Related articles” function within Google Scholar. Clinical trial databases were not accessed.

The inclusion criteria were cohorts of patients requiring prolonged (>24 h) ICU stays for which the incidence proportions of VAP overall and VAP in association with Staphylococcus aureus, Pseudomonas aeruginosa, and Acinetobacter species. The listing of at least two of the three infections was required so as to avoid selecting studies which may have reported only the most prominent VAP isolate type. Data were extracted for each component group where there was more than one type of control or intervention group.

The studies were classified into three broad groups of study interventions being non- antimicrobial, antiseptic-based, and antibiotic-based. Non-antimicrobial interventions were studies of various approaches to the control of upper gastrointestinal tract colonization through various stress ulcer prevention or feeding approaches, as well as various approaches to control airway colonization through airway management [90,91,92,93,94,95].

Antiseptic-based interventions included the use of agents such as chlorhexidine, povidone-iodine and iseganan. All antiseptic exposures were included regardless of whether the application was to the oropharynx, by toothbrushing, or by body wash [98,99]. Antibiotic-based interventions included the use of either topical antibiotic prophylaxis (TAP) to the oropharynx or stomach (without regard to the specific antibiotic constituents) or whether protocolized parenteral antibiotic prophylaxis (PPAP) was used in addition to the topical antibiotics or used alone [100,101].

The inclusion criteria were deliberately broad without regard to the frequency or duration of interventions under study or any criteria of study quality. Studies published between 1985 and 2024 were included. Studies originating from pediatric ICUs were not excluded. Studies for which patient inclusion was on the basis of risk factors for fungal infections and studies with fewer than 25 patients were excluded.

2.2. Outcomes of Interest

Regarding the count of VAP, both overall and in association with each of Staphylococcus aureus, Pseudomonas aeruginosa, and Acinetobacter species, there was no imputation of missing data. Study quality was taken as judged in each source document and standardized into a simple binary score based on whether the quality score was assigned a majority of possible scoring points.

The independent variable in the regression models was the mean length of the ICU stay (LOS). If this was not available, the median LOS or the mean or median duration of mechanical ventilation were used. The VAP incidence proportion and LOS data were all derived from the original publications. The VAP incidence proportion, being count data, were logit transformed using the number receiving prolonged (>24 h) mechanical ventilation as the denominator. The LOS data were positively skewed and were log transformed.

2.3. Summary Effect Size; Contrast-Based Analysis

Indicative summary prevention effect sizes versus VAP, for each category, were derived from all studies. Summary prevention effect sizes versus VAP in association with Staphylococcus aureus, Pseudomonas aeruginosa, and Acinetobacter species were also derived where available.

Study specific and overall summary VAP prevention effect sizes and associated 95% confidence interval were calculated for each category using random effect methods of meta-analysis. The data were set for two group comparisons of binary outcomes using the ‘meta’ command in Stata 18 (Stata Corp.; College Station, TX, USA) and the default restricted maximum likelihood [124].

2.4. Arms-Based Analysis and SROC Plots

The data from the component control and intervention groups of randomized concurrent controlled trials were decanted from each randomized concurrent controlled trial with care to include the groups from the three arm studies only once. Summary VAP incidence proportion data were derived using the ‘meta’ command in Stata 18 and the default restricted maximum likelihood. SROC plots, being a bivariate plot of control versus intervention group incidences of VAP, were derived [106,107].

2.5. Arms-Based Meta-Regression

The relationship between study specific prevention effect sizes versus log transformed LOS of each of the three broad categories of intervention toward the prevention of VAP were modelled by meta-regression.

2.6. Availability of Data and Materials

All data generated or analyzed during this study have been included in this published article (see Table 1).

3. Results

3.1. Characteristics of the Studies

Of the 88 randomized concurrent controlled trials identified by the search, 48 studies were found in either twelve Cochrane [90,91,92,93,94,95,96,97,98,99,100,101] or other systematic reviews [102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,125]. Most studies were published between 1990 and 2010, and most had a mean ICU LOS exceeding ten days. Five randomized concurrent controlled trials had either more than one type of intervention group or more than one type of control group. Most groups had between 50 and 100 patients. Most studies originated from either North American or European ICUs. VAP count data for all three VAP isolates were available from 58 randomized concurrent controlled trials (Table 2).

The most common non-antimicrobial interventions were gastric pH management (10 studies), heat and moisture exchange (11 studies) and endotracheal tube management (8 studies). The most common antiseptic intervention was topical chlorhexidine (4 studies), and the most common antibiotic intervention was some type of combination topical antibiotic and protocolized parenteral antibiotic prophylaxis regimen (11 studies).

3.2. Prevention Effect Sizes

The study specific and summary effect sizes derived by contrast-based and arms-based methods for the three categories of intervention against VAP are presented in Table 3. Significant summary prevention effects against both overall VAP and S. aureus VAP were evident for all three categories. There were significant summary prevention effects against Pseudomonas VAP for the non-antimicrobial and antibiotic categories.

The summary effect size estimates derived by arms-based methods, as diagnostic odds ratios, were similar to those derived by contrast-based methods for overall VAP, Pseudomonas VAP, and S. aureus VAP. The SROC models failed to converge with the Acinetobacter for the non-antimicrobial and antiseptic models.

SROC plots of summary effect size of non-antimicrobial (a; top), antiseptic (b; middle), and antibiotic (c; bottom) interventions in preventing overall VAP. Pneumonia incidence among control and intervention groups with symbol size proportional to group size (hollow circles). The diagonal dotted line is the line of equivalence and the curved green line is the summary SROC curve. Also shown are the summary point (solid red square), the hierarchical summary ROC curve (green) with 95% confidence limits (dotted yellow inner ellipse), and 95% prediction limits (dotted purple outer ellipse).

SROC plots of summary effect size of non-antimicrobial (a; top), antiseptic (b; middle), and antibiotic (c; bottom) interventions in preventing Pseudomonas VAP. Pneumonia incidence among control and intervention groups with symbol size proportional to group size (hollow circles). The diagonal dotted line is the line of equivalence and the curved green line is the summary SROC curve. Also shown are the summary point (solid red square), the hierarchical summary ROC curve (green) with 95% confidence limits (dotted yellow inner ellipse), and 95% prediction limits (dotted purple outer ellipse).

SROC plots of summary effect size (and associated 95% confidence and 95% prediction ellipses) of non-antimicrobial (a; top), antiseptic (b; middle), and antibiotic (c; bottom) interventions in preventing Staphylococcus aureus VAP. The diagonal dotted line is the line of equivalence and the curved green line is the summary SROC curve. Also shown are the summary point (solid red square), the hierarchical summary ROC curve (green) with 95% confidence limits (dotted yellow inner ellipse), and 95% prediction limits (dotted purple outer ellipse).

3.3. Arms-Based Analysis and Meta-Regression

The summary overall VAP incidences for the control groups of the studies of antibiotic interventions, being 34%, were higher versus the summary incidences for the control groups for the other two categories, being 21 and 22%. Paradoxically, the summary overall VAP incidences for the intervention groups, being in the range of 12 to 16%, were similar across the three categories (Table 3).

The mean overall VAP proportion was above 40% for 11 of 27 antibiotic control groups but only six of the 63 control groups in the other studies. The mean overall VAP proportion was below 5% for only 11 intervention groups, eight from non-antimicrobial studies and three from studies of antibiotic-based interventions (Figure 2).

Likewise, the summary Pseudomonas VAP and Staphylococcus aureus VAP incidences for the control groups of the studies of antibiotic interventions were higher versus the summary VAP incidences for the control groups of the other two categories. Again, paradoxically, the summary VAP incidences for these isolates for the intervention groups for the three categories were more similar to each other (Table 3; Figure 3 and Figure 4).

These paradoxical VAP incidences were also apparent in the arms-based meta-regression whether with or without adjustment for LOS (Table 4). Membership in a concurrent control group of a study of an antibiotic-based intervention was associated with a significantly higher incidence of all types of VAP. Membership in an intervention group of a study of an antibiotic-based intervention was not associated with a lower incidence of overall VAP, although it was associated with a lower incidence of Pseudomonas VAP in the adjusted model.

4. Discussion

With respect to the four objectives here, firstly, the effect size estimates for three broad categories of prevention interventions against overall VAP derived using contrast-based versus arms-based meta-analysis methods here were substantially equivalent.

Secondly, these prevention effect size estimates triangulated with previous effect size summaries for the three broad categories of prevention interventions derived for a larger number of studies within the Cochrane reviews and other literature sources (Table 5). The summary prevention effect sizes of the various non-antimicrobial interventions derived for a larger number of studies within the literature generally had risk ratios between 0.5 and 0.95, which compares to the odds ratio derived here from all 56 RCCTs of non-antimicrobial interventions of 0.73 (95% confidence interval 0.61 to 0.86).

Third, the prevention effect estimates for the various interventions differed slightly in their prevention effect towards VAP associated with each of Staphylococcus aureus, Pseudomonas aeruginosa, and Acinetobacter species. However, with each, the odds ratios derived from contrast-based versus arms-based meta-analysis methods were substantially similar (Table 3). Moreover, the estimates obtained in the meta-regression models were robust to adjustment for group mean LOS, quality score, and year of publication (Table 4). In all models, membership in a concurrent control group of an RCCT of an antibiotic-based VAP prevention intervention were strong and positive predictors for each of the associated VAP incidences; by contrast, membership in an intervention group of an RCCT of an antibiotic-based VAP prevention intervention were weaker and generally insignificant negative predictors.

Finally, in the arms-based analysis and SROC plots, VAP-zero appeared to be elusive for both VAP overall and VAP in association with each of Staphylococcus aureus, Pseudomonas aeruginosa, and Acinetobacter species. For example, incidences for overall VAP of less than 5%, the lower limit of the expert VAP incidence range, were rare (Figure 2). Moreover, low VAP incidences were no more common among studies of antimicrobial versus non-antimicrobial interventions either for overall VAP or for VAP with any of the specific microbial types.

The mean VAP incidence and mean Pseudomonas aeruginosa-associated VAP incidence estimated here were similar to that observed in a worldwide multi-center prospective study [127].

The strengths in the analysis here included the large number of studies reporting at least two VAP types. Studies with only one VAP type might have either reported only the most prominent VAP type or the study may have had a specific focus on that VAP type [128]. This may have created a potential reporting bias.

The principle underlying contrast-based methods is that the random assignment of intervention, which is central to causal inference from RCCTs, is retained in the meta-analysis model [129]. Hence, the fundamental criticism of arms-based methods was that it ‘breaks’ this random assignment [130]. On the other hand, if there was spillover, the stable unit treatment value assumption, which was central to causal inference originating from random assignment, is no longer tenable. In which case, an arms-based analysis is to be preferred [131]. Moreover, arms-based analysis can enable comparisons of event rates observed within each arm to external benchmarks, such as the expert opinion VAP incidence range, towards deriving population level inferences.

The analysis extended the novel use of SROC methods as an arms-based approach to the analysis of RCCT data [107,108]. The SROC method was recently adapted for application to the meta-analysis of diagnostic tests. The SROC plot resembled the L’abbe plot as derived within a meta-analysis of RCCTs. Each displayed the dispersion in event rates in the two component groups along the y-axis for one versus the x-axis for the other [124]. For the L’abbe plot, these were the event rates in the intervention versus control groups, respectively. For the SROC plot, these were the test positive rates among the diseased (sensitivity) versus the non-diseased (which equates to 1 minus specificity), respectively. In both cases, the diagonal (y = x line) represented the locus where the event rates in the two populations in the comparison were equal. The two plots differed in how the covariation away from this line was displayed and how event rate dispersion was inferred. For the L’abbe plot, depending on whether the ES was defined as an odds ratio (OR), a risk ratio (RR), or a risk difference (RD), this gave a visual representation of covariation as either a line parallel to the y = x line (RD), a line that passed through the origin (RR), or a curve (OR), respectively. For the L’abbe plot, dispersion was merely a subjective visual inference which was dependent on whether the presumptive underlying relationship was an RD, RR, or OR.

For the SROC plot, on the other hand, the underlying relationship was always displayed as an OR and the dispersion in event rates, quantified as a summary point together with an enveloping 95% prediction ellipse, enabling projections of the sensitivity and specificity that future applications of the diagnostic test of interest might experience. The SROC displayed the summary operating curve which mapped the summary values of sensitivity and specificity within the SROC plot. Moreover, instead of two unidirectional 95% confidence limits, these models provided bi-directional 95% confidence regions (as ellipses) rather than as together with 95% prediction ellipses [108].

By displaying the event rates in both the control and intervention arms, arms-based methods can accommodate the potential issue of spillover effects to concurrent patients within the ICU not receiving decontamination as a possible ‘driver’ of the whole of intensive care unit infection event rates [107].

Antimicrobial-based interventions, using either topical antiseptics and oral care [98,99] or antibiotics [100,101], were presumed to alter the microbiome of the entire ICU. This spillover of intervention effect was anticipated from the first study [132] being postulated as “…having heavily contaminated patients next to decontaminated patients might adversely affect the potentially beneficial results [postulate one]. Secondly, a reduction of the number of contagious patients by applying [selective digestive decontamination] SDD in half of them, might reduce the acquisition, colonisation and infection incidence in the not-SDD-treated control group [postulate two]” [132].

Surprisingly, the randomized controlled trials of antimicrobial-based decontamination interventions with concurrent controls had an overall VAP incidence which was higher, not lower as was postulated above, than what might be expected in comparison to expert VAP incidence range estimates [109,110,111,112,113,114,115]. This high VAP remains unexplained. These RCCTs also had higher incidences of blood stream infections [102], candidemia [114], and mortality [133] which likewise are unexplained.

Rebound infection on withdrawal of antibiotic-based infection prevention interventions also need to be considered. Rebound infection had been noted among patients that became neutropenic following cytotoxic chemotherapy in hematology units in the 1970s. These severe and occasionally fatal infections were observed in patients who had prematurely discontinued the antibiotic-based intervention due to its intolerable taste. Rebound sepsis has been noted following hospital discharge among patients exposed to antibiotic therapy considered high risk for causing microbiome disruption [134].

Rebound may be imperceptible without specific surveillance for colonization and infections on withdrawal of decontamination interventions. Rebound following antibiotic-based discontinuation and ICU discharge manifested as a 50% increased risk of hospital-acquired infection [135]. Rebound of ceftazidime resistant Gram-negative bacteria may occur as a ‘whole of ICU’ phenomenon not limited to the antibiotic-based recipients, persisting as an ecological effect for several months after antibiotic-based withdrawal [136,137].

The use of protocolized parenteral antibiotic prophylaxis within some concurrent control groups may have modified the rebound and spillover effects from the intervention groups within these RCCTs.

There was an uncertain amount of spillover effect in these concurrent controlled RCCTs of VAP prevention using antimicrobial-based interventions which was as previously noted for several end points [138,139]. Any spillover effect would conflate the apparent prevention effect [140] which underlie the paradoxical observations [141].

Limitations

Several limitations should be considered.

There was considerable heterogeneity in the interventions, populations, study quality and RCCT designs among studies published over several decades included in the analysis here with no ability to adjust for underlying patient risk. The definitions of VAP used in various studies varied and this further added to the heterogeneity for endpoints related to VAP incidence. Despite this heterogeneity in overall VAP incidences, incidences < 5%, the lower limit of the expert VAP incidence range, were rare.

Whilst the RCCTs included here generally rated highly for study quality within Cochrane reviews, the potential effects of spillover and rebound were not recognized in these quality scores. Hence, the prevention effect estimates were considered ‘indicative’, and primarily related to the population level rather than the patient level of analysis.

The literature search has been opportunistic rather than systematic. By using existing systematic reviews as a starting point, the key interventions from the broadly selected studies can be readily identified and classified. These Cochrane reviews served as a source of effect size estimates for triangulation.

Mean LOS and even median LOS were crude measures of group level exposure for each group in the ICU context and exposure to the infection prevention interventions in the intervention groups. Of note, even cohorts with short mean LOS will contain patients with long LOS and vice versa. The analysis was ecological, and the estimates related to the impacts of antiseptic-based and antibiotic-based interventions on ICU patient cohorts. The associations for group-wide exposures may not equate to associations at the patient level of exposure.

Many studies of decontamination interventions will have been underpowered to adequately assess key safety end points or to assess for novel microbiome interactions that have been thought to contribute to VAP [116].

There has been no imputation of missing data. There has been no search for data outside of the English-language literature. Only the three major subgroups have been analyzed with no further subgroup analysis due to the limited number of studies.

Finally, there could be the potential for publication and reporting bias if studies merely listed only the most prominent VAP isolate. This bias has been addressed by limiting inclusion to those studies reporting at least two isolates. However, this leaves a substantial number of studies which failed to report the VAP isolates. Also, uncommon isolates that might be causes of VAP have not been considered, such as Enterococci [139]. Of note, the effect size metrics for the prevention of overall VAP estimates here were similar to those derived from all available studies whether or not a listing VAP isolate data was available.

5. Conclusions

VAP-zero both overall and in association with specific microbial sub-types remains elusive using antimicrobial-based interventions. The control group incidence of VAP in association with Staphylococcus aureus, Acinetobacter, and Pseudomonas aeruginosa was unusually high among RCCTs that showed apparent effectiveness of antimicrobial-based VAP prevention interventions.