Validation of the prevalence to incidence conversion method for healthcare associated infections in long-term care facilities

TL;DR

This study tested if a method for converting HAI prevalence to incidence works in long-term care facilities and found it to be valid.

Contribution

The study validates the ECDC's prevalence-to-incidence conversion method in LTCFs, where prior assumptions suggested potential underestimation.

Findings

HAI prevalence from PPS was 1.47%, and incidence from longitudinal study was 3.53 per 1000 patient-days.

Converted prevalence-based incidence was 0.86 per 1000 patient-days, with a non-significant rate ratio of 0.24.

No significant difference was found between incidence estimates from the two methods.

Abstract

Residents of long-term care facilities (LTCFs) are a population at high risk of developing severe healthcare associated infections (HAIs). In the assessment of HAIs in acute-care hospitals, selection bias can occur due to cases being over-represented: patients developing HAIs usually have longer lengths of stays compared to controls, and therefore have an increased probability of being sampled in PPS, leading to an overestimation of HAI prevalence. Our hypothesis was that in LTCFs, the opposite may occur: residents developing HAIs either may have a greater chance of being transferred to acute-care facilities or of dying, and therefore could be under-represented in PPS, leading to an underestimation of HAI prevalence. Our aim was to test this hypothesis by comparing HAI rates obtained through longitudinal and cross-sectional studies.

Results from two studies conducted simultaneously in four LTCFs in Italy were compared: a longitudinal study promoted by the European Centre for Disease Prevention and Control (ECDC, HALT4 longitudinal study, H4LS), and a PPS. Prevalence was estimated from the PPS and converted into incidence per year using an adapted version of the Rhame and Sudderth formula proposed by the ECDC. Differences between incidence rates calculated from the PPS results and obtained from H4LS were investigated using the Byar method for rate ratio (RR).

On the day of the PPS, HAI prevalence was 1.47% (95% confidence interval, CI 0.38–3.97), whereas the H4LS incidence rate was 3.53 per 1000 patient-days (PDs, 95% CI 2.99–4.08). Conversion of prevalence rates obtained through the PPS into incidence using the ECDC formula resulted in a rate of 0.86 per 1000 PDs (95% CI 0–2.68). Comparing the two rates, a RR of 0.24 (95% CI 0.03–2.03, p 0.1649) was found.

This study did not find significant differences between HAI incidence estimates obtained from a longitudinal study and through conversion from PPS data. Results of this study support the validity of the ECDC method.

Figures

Lexis diagrams illustrating the study hypothesis.Diagonal lines represent the length of stay of each patient. Grey lines represent length of stay (LOS) without infection. Black dots represent the occurrence of a healthcare-associated infection (HAI), and black lines represent the duration of infection. Vertical blue lines represent the day of the point prevalence survey (PPS), in which patients are sampled. The study hypothesis is that compared to acute-care settings, where patients with HAIs are over-represented in PPS due to their longer LOS, residents of long-term care facilities developing HAIs are under-represented, as they may have a greater chance of being transferred to acute-care hospitals (ACH) or of dying (red dots).

Diagram illustrating the study design.HAI, healthcare-associated infection; H4LS, healthcare-associated infections and antimicrobial use in long-term care facilities longitudinal study; LTCFs, long-term care facilities; PPS, point prevalence survey.

Full text

Introduction

Residents of long-term care facilities (LTCFs) are a population at high risk of developing severe healthcare associated infections (HAIs), as they represent an elder and frail population, often affected by numerous comorbidities and with high healthcare exposure, residing in settings with limited infection prevention and control (IPC) resources [1]. HAIs in LTCF residents are associated with increased risk and length of hospitalization, increased risk of death, and increased healthcare costs [2]. These aspects have implications on the broader health system, not only in terms of the number of patients requiring acute care. LTCFs can act as reservoirs for antimicrobial resistant (AMR) pathogens, and residents transferring across facilities providing different levels of care have the potential of amplifying the spread of AMR pathogens throughout a health system, as seen for Candida auris [3]. Further, the treatment of HAIs acquired in LTCFs is a growing indication for antibiotic use in acute-care facilities, which in turn increases antibiotic pressure [4].

Assessing the burden of HAIs requires therefore an integrated approach, including both acute-care settings and LTCFs. The European Centre for Disease Prevention and Control (ECDC) coordinates repeated point prevalence surveys (PPS) of HAIs across European countries in both settings. Based on results of the third Healthcare-Associated Infections and Antimicrobial Use in Long-Term Care Facilities (HALT-3) project, over 4 million HAI episodes among residents of European LTCFs were estimated to have occurred in 2016, amounting to 129,940 residents with at least one HAI on a given day [5].

Incidence estimates obtained from PPS data have been used to assess the burden of HAIs in acute-care hospitals in terms of disability-adjusted life years (DALYs) [6, 7]. Measuring disease burden in DALYs allows a more comprehensive evaluation compared to mortality, as DALYs add years lived with disability (YLD, with a specific disability weight attributed to each health condition) to years of life lost (YLL). Further, calculating DALYs allows to compare disease burdens of very different health conditions, including but not limited to other communicable diseases. Calculating DALYs using an incidence-based approach allows a more consistent measurement of YLL and YLD, and is generally considered more appropriate for acute conditions, such as infectious diseases [8]. However, longitudinal studies of HAI incidence are relatively more resource-intensive compared to PPS, which is particularly true for the LTCF setting. In fact, very few HAI incidence studies conducted in LTCFs have been published to date [9, 10].

For these reasons, DALYs have been calculated by converting HAI prevalence obtained from PPS into incidence, using a modified version of the Rhame and Sudderth formula [11]. This method requires a certain degree of approximation due to the cross-sectional nature of PPS: in particular, residents are assumed to permanently reside in LTCFs, and HAI duration is assessed by multiplying by 2 the median duration from day of onset to the day of the PPS for each specific HAI type [5]. Further, in the assessment of HAIs in acute-care hospitals, selection bias can occur due to cases being over-represented: in fact, patients developing HAIs usually have longer lengths of stays compared to controls, and therefore have an increased probability of being sampled in PPS, leading to an overestimation of HAI prevalence [12, 13]. Our hypothesis was that in LTCFs, the opposite may occur: residents developing HAIs either may have a greater chance of being transferred to acute-care facilities or of dying, and therefore could be under-represented in PPS, leading to an underestimation of HAI prevalence (Fig 1).

Lexis diagrams illustrating the study hypothesis.Diagonal lines represent the length of stay of each patient. Grey lines represent length of stay (LOS) without infection. Black dots represent the occurrence of a healthcare-associated infection (HAI), and black lines represent the duration of infection. Vertical blue lines represent the day of the point prevalence survey (PPS), in which patients are sampled. The study hypothesis is that compared to acute-care settings, where patients with HAIs are over-represented in PPS due to their longer LOS, residents of long-term care facilities developing HAIs are under-represented, as they may have a greater chance of being transferred to acute-care hospitals (ACH) or of dying (red dots).

To test this hypothesis, we compared results from two studies conducted simultaneously in four LTCFs in Italy: a longitudinal study promoted by the ECDC (HALT4 longitudinal study, H4LS), and a pilot PPS, which applied the same HAI definitions. Our objective was to validate the prevalence to incidence formula proposed by the ECDC in the LTCF setting, to evaluate whether resulting rates would be underestimated in comparison with HAI rates observed in the longitudinal study.

Material and methods

Data sources

Data were obtained from LTCFs participating simultaneously in two studies, respectively cross-sectional (pilot PPS) and prospective cohort (H4LS). Both studies were based on the HALT project and applied the same ECDC HAI definitions. Participation was voluntary, and convenience sampling was employed in both studies.

The pilot PPS was conducted between May 15—June 15, 2022. The study was coordinated by the University of Turin in collaboration with the National Health Institute (Istituto Superiore di Sanità, ISS) within a broader project aiming to establish a national HAI surveillance network, financed by the Italian Ministry of Health. The specific objective of the pilot PPS was to establish a surveillance network and to test data collection methods and tools. The study was previously described in detail [14]. Briefly, data were collected in a single day per participating facility at the LTCF, ward, and resident levels. Each facility could pick a date to conduct the PPS within the study window. At the resident level, all active HAIs were recorded, defined as conditions for which signs/symptoms of infection were present on the day of the PPS or for which antibiotic treatment was still ongoing on the day of the PPS.

For the longitudinal study, the H4LS protocol version 1.5 was applied [9]. The study was promoted by the ECDC through the healthcare-associated infections surveillance network (HAI-Net). The national study coordinator for Italy was the Region of Emilia-Romagna. Participating LTCFs were required to prospectively collect data for 12 months, starting January 31^st^, 2022. LTCFs with a median length of stay of 12 months or more were preferred. Within participating facilities, residents present on day 1 of the study were included. Data were collected through institutional, resident, and infection questionnaires. The first two questionnaires were completed at the beginning of the study (including resident clinical characteristics, comorbidities, and risk factors), whereas infection questionnaires were completed for each HAI developed during the follow-up period.

For the purposes of this analysis, data from four LTCFs participating in both studies (PPS and H4LS) were considered. A diagram summarizing the study design is provided in Fig 2.

Diagram illustrating the study design.HAI, healthcare-associated infection; H4LS, healthcare-associated infections and antimicrobial use in long-term care facilities longitudinal study; LTCFs, long-term care facilities; PPS, point prevalence survey.

Institutional review boards

Both studies were approved by relevant institutions. The pilot PPS was approved by the “Comitato di bioetica d’Ateneo, University of Turin” (protocol n. 0169983, 14 March 2022), “Comitato Etico Interaziendale, Azienda Ospedaliero Universitaria San Luigi Gonzaga, Orbassano” (protocol n. 4611, 21 March 2022) and “Comitato Etico Nazionale per le sperimentazioni degli Enti Pubblici di Ricerca e altri Enti Pubblici a carattere nazionale, Istituto Superiore di Sanità” (protocol n. 0015064, 19 April 2022), in addition to approvals by each appropriate local institutional review board. H4LS received approvals from the “Comitato Etico Interaziendale, Azienda Ospedaliero Universitaria San Luigi Gonzaga, Orbassano” (protocol n. 2720, 18 February 2022) and the “Comitato Etico Area Vasta Emilia Centro della Regione Emilia-Romagna (CE-AVEC)” (protocol n° 929-2021-OSS-AUSLIM, 17 November 2021).

The recruitment period for the pilot PPS coincided with the study period (May 15—June 15, 2022). For the pilot PPS, the need for consent was waived by the ethics committee. Residents were recruited for the longitudinal study from December 1^st^, 2021 to January 31^st^, 2022. Written consent for participation was required from either residents or their caregivers for participation in H4LS.

Statistical analysis

Descriptive analyses were performed on structure- and patient-level data, using frequencies and medians with interquartile ranges (IQR) for categorical and continuous variables respectively.

Characteristics of residents included in H4LS which developed at least one HAI were compared to those of patients that did not develop infection during the study period, using Fisher and Mann-Whitney U tests when appropriate.

Prevalence was estimated from the pilot PPS in terms of overall prevalence (total number of residents with at least an HAI on the day of the PPS divided by all included residents). Prevalence was converted into incidence per year using an adapted version of the Rhame and Sudderth formula proposed by the ECDC:

$$[eqn]$$

where I is incidence, P is prevalence, D is the duration of infection (in days), calculated as the median number of days between date of HAI onset and date of the PPS. The formula implies residents are assumed to reside permanently in LTCFs. A correction factor of 95% is applied to account for average occupancy of LTCF beds [5, 11]. The correction factor was calculated by ECDC based on institutional denominator data collected during HALT-3 [5]. Incidence rates were expressed in HAIs per 1000 patient-days (PDs), and respective 95% confidence intervals (CIs) were obtained using Taylor series.

Considering H4LS data, the incidence rate was expressed as the number of HAI episodes over 1000 days of follow-up (PDs), with 95% CI calculated according to the normal approximation to the Poisson distribution, as described by Rosner [15]. Differences between incidence rates calculated from the pilot PPS results and obtained from H4LS were investigated using the Byar method for rate ratio (RR) [16]. Analyses were performed using IBM SPPS v. 28.0.1, setting a two-tailed significance level at α = 0.05.

Results

Descriptive statistics

Overall, 15 LTCFs participated in the pilot PPS conducted in May-June 2022. The Italian H4LS sample included 24 LTCFs, and was conducted between January 2022 –March 2023. Four LTCFs (two in the region of Piedmont, and two in the region of Emilia-Romagna) participated in both studies and were therefore included in our analysis. Facility-level descriptive characteristics for each of the LTCFs are provided in Table 1.

Table 1: Structure-level characteristics from long-term care facilities (LTCFs) participating in both the pilot point prevalence survey (PPS) and longitudinal study (H4LS) of healthcare-associated infections (HAIs).

Two of the facilities were classified as residential homes: residents are unable to live independently, and require supervision and assistance for the activities of daily living. The other two facilities were mixed LTCF, providing different types of care within the same facility. The proportion of residents included in the pilot PPS ranged from 67% to 100%, and from 26.3% to 88.3% for H4LS. Reasons for exclusion were mainly due to difficulties in obtaining informed consent from the residents themselves or from their care-givers. IPC resources and practices were uneven, with only one LTCF employing personnel trained in IPC. However, all LTCFs reported having an IPC committee and routinely performing laboratory tests to diagnose infections.

Resident-level characteristics from H4LS are provided in Table 2. Residents developing at least one HAI during follow-up had a higher median Charlson score compared to residents which did not develop HAI, and required hospitalization more frequently (15.2% vs. 4.2%). Hospitalizations among HAI residents also lasted longer. On the other hand, at end of follow-up a higher proportion of patients not developing HAIs were deceased compared to patients developing HAIs, which was the only difference reaching statistical significance among investigated variables (p 0.030).

Table 2: Resident-level characteristics and outcomes from the HALT4 longitudinal study (H4LS), stratified according to the occurrence of healthcare-acquired infections (HAIs) during the study period; N = 147.

Characteristics of HAIs are summarized in Table 3. On the day of the pilot PPS, the overall prevalence of HAI was 1.47% (95% CI 0.38–3.97), whereas 67.3% of residents included in H4LS developed at least one HAI during the year of follow-up. Overall, the total number of HAIs recorded during H4LS was 160, resulting in an overall incidence rate of 3.53 per 1000 PDs (95% CI 2.99–4.08). Repeated infections were frequent among patients developing at least one HAI.

Table 3: Characteristics of healthcare-associated infections (HAIs) recorded through the pilot point prevalence survey (PPS) and longitudinal study (H4LS).

Concerning HAI types, 1 skin infection, 1 ear, eye, nose & mouth infection, and 1 other infection were recorded on the day of the pilot PPS. The most frequent HAI types in H4LS were COVID-19, lower respiratory tract infections, and urinary tract infections. Over half of HAIs in the longitudinal study underwent microbiological testing (for none of which antimicrobial susceptibility testing results were available), and none in the pilot PPS.

According to H4LS results, the median duration of HAI episodes was 7 days (IQR 5–10). The median number of days from HAI onset to the day of the PPS was 9 (range 7–19). In the longitudinal study, 11 HAIs led to residents being hospitalized (6.7% of recorded HAIs) and in one case infection was either part of the causal sequence, contributory cause or sole cause of death (0.6% of HAIs).

Validation of prevalence to incidence conversion

The calculation of HAI incidence from overall prevalence resulted in a rate of 0.86 per 1000 PDs (95% CI 0–2.68). Comparing this result to HAI incidence obtained from H4LS resulted in a RR of 0.24 (95% CI 0.03–2.03, p 0.1649). Even though this result did not reach statistical significance, the prevalence to incidence conversion method resulted in an underestimation by 1/4 of HAI incidence.

A possible cause for the underestimation of HAI incidence could be due to an overestimation of the duration of HAIs in the cross-sectional survey: in fact, median HAI duration estimated from the pilot PPS was 2 days longer than the duration observed in H4LS. We performed the same calculation using the median HAI duration from H4LS: an incidence rate of 2.22 per 1000 PDs (95% CI 0–5.14) was found, a result much closer to H4LS incidence (RR 0.63, 95% CI 0.17–2.36, p 0.451).

To correct for the overestimation of HAI duration, we propose the following formula for prevalence to incidence conversion:

$$[eqn]$$

Applying this formula to pilot PPS data (prevalence and HAI duration), we found an incidence rate of 1.73 per 1000 PDs (95% CI 0–4.31), which compared to H4LS incidence resulted in a RR of 0.49 (95% CI 0.11–2.19, p 0.451).

We hypothesized that HAIs with a longer duration have a higher probability of being recorded in PPS compared to shorter HAIs. As we could evaluate the same HAIs through both the longitudinal and cross-sectional survey, we evaluated the probability of HAIs occurring during H4LS of being recorded in the pilot PPS, according to HAI duration (Table 4). None of the HAIs with a duration under 5 days were recorded in the pilot PPS, whereas an increasing probability was found for progressively longer HAIs: 2.08% for HAIs lasting 5–10 days and 2.78% for HAIs lasting > 10 days.

Table 4: Probability of healthcare-associated infections (HAIs) occurring during the longitudinal survey (H4LS) of being recorded in the pilot point prevalence survey (PPS), stratifying HAIs according to duration.

Discussion

HAIs in LTCF are a relevant public health and patient safety issue, with potential ramifications throughout the broader health system. This study provided data on HAIs among LTCF residents measured simultaneously through two different study designs, a cross-sectional and a longitudinal study, which applied similar definitions and standardized methods for data collection. Cross-sectional studies allow to estimate prevalence proportions, and by design lack prospective follow-up. Longitudinal designs allow to estimate incidence rates, however require significantly more time and resources [17]. Considering the limited IPC resources of the LTCF setting in particular, this study provided the rare opportunity to validate the methodology proposed by the ECDC for prevalence to incidence conversion, which is an important step in burden estimations [6, 7, 9]. Further, through this study it was possible to formulate hypotheses on the underlying reasons for potential discrepancies between measured incidence and estimates obtained through conversion of prevalence proportions.

Our primary hypothesis for this study was that in the LTCF setting, length-biased sampling seen in acute-care may occur in the opposite direction: residents developing HAIs may be under-represented in PPS, due to a greater chance of being transferred to acute-care facilities or of dying, leading to underestimated HAI prevalence estimates [12]. Comparing HAI incidence obtained from H4LS with the estimate calculated from HAI prevalence from the pilot PPS, no significant difference was found, supporting the validity of the ECDC method. However, the lack of a significant difference could be due to the small number of events seen in the longitudinal study, which is in line with previous reports. An Austrian study with a similar design and which applied definitions based on the ECDC HALT project found an HAI incidence of 2.1 episodes per 1000 patient-days in 2018 (COVID-19 cases were not included) [10].

Regardless of significance, the prevalence to incidence conversion method resulted in an underestimation of one third of HAI incidence, which would have an important impact on burden estimates. A possible explanation could be the overestimation of HAI duration in PPS. As aforementioned, PPS do not allow to gather prospective information. The ECDC method assumes that the time interval between HAI onset and date of the PPS represents 50% of HAI duration. However, as suggested by results of this study, longer HAIs may have an increased probability of being recorded in PPS. Therefore, we propose the simple solution of slightly altering the conversion formula, by only taking into account the length of infection actually measured through the PPS.

When applying the corrected formula, we still found HAI incidence remained underestimated by 50%. Another solution could be applying inverse probability weighting, as proposed for correcting length-biased sampling in PPS of HAIs in acute-care hospitals [12]. Further, Doerken et al. have previously suggested that PPS of HAIs in acute-care hospitals should also collect some follow-up data from included patients, namely any incident HAIs and discharge dates [17]. Prospectively collecting data on HAI duration could also improve estimates, and would require relatively less efforts compared to conducting a longitudinal study.

The main limitation of this study was the limited sample size. The small sample size is a concern in the interpretation of the results, affecting generalizability, the lack of precision and reliability and the risk of random variability. Further, participation of LTCFs in both projects was voluntary, therefore we cannot exclude selection bias and make no claim of the representativeness of our results. Another important limitation was the requirement of informed consent for participation in the longitudinal study, which may have introduced additional selection bias.

In conclusion, this study did not find significant differences between HAI incidence estimates obtained from a longitudinal study and through conversion from PPS data. Given the importance of burden estimates in informing policy-makers on rational allocation of healthcare resources, we hope to have contributed to achieving more accurate measurements of HAI incidence in LTCF settings. Further studies applied in a larger sample size could provide more reliable and accurate results.