Demystifying Quality Metrics and Unveiling the True Measure of Quality of Care in Nursing Homes: Mixed Effects Analysis

TL;DR

This study evaluates how well the 5-Star Quality Rating System for nursing homes predicts care quality during the pandemic, finding it unreliable.

Contribution

The study introduces a mixed effects analysis framework to assess the validity of nursing home quality metrics during the COVID-19 pandemic.

Findings

Staffing measures and vaccination rates strongly correlate with better pandemic outcomes in nursing homes.

The 5-Star Quality Rating System shows no clear relationship with resident or staff infection/death rates.

Health deficiency scores and hospitalization rates are significant predictors of pandemic performance.

Abstract

The 5-Star Quality Rating System for nursing homes plays a central role in evaluating quality of care, although it has both strengths and limitations. This system relies heavily on the Minimum Data Set and derives several quality measures (QMs) from it. In this study, we validated the effectiveness of the 5-Star Quality Rating System for nursing homes and its underlying QMs in estimating quality of care. We constructed a panel dataset of US nursing homes (n=15,416) active from May 2020 to June 2023, retrieving data from three major sources: (1) COVID-19 nursing home data, (2) Payroll-Based Journal data, and (3) nursing home QM snapshots. The outcome variables included (1) resident infection, (2) staff infection, or (3) resident and staff deaths. The predictor variables were the 5-Star Quality Rating System for nursing homes and its underlying QMs classified as structure, process, or outcome (SPO) QMs.

This study aims to evaluate the effectiveness of nursing home QMs by regressing nursing home COVID-19 outcomes on nursing home QMs, classified using the Donabedian SPO framework. We hypothesized that nursing homes with better structural quality (eg, greater staff availability, better skill mix, and so on), better process quality (eg, lower restraint use and higher vaccination rates), and better outcome quality (eg, lower number of residents with pressure ulcers and a lower number of resident falls) experienced better COVID-19 performance in terms of resident and staff infections and deaths.

To examine the association between the COVID-19 outcomes and SPO QMs, we imputed missing values in the dataset using random forest. Subsequently, we modeled the imputed dataset using hurdled zero-inflated negative binomial mixed effects models. The zero inflation model included factors influencing initial susceptibility to COVID-19 or factors influencing the possibility of death after COVID-19 had been contracted. The model estimates were conditioned on zero inflation and random effects.

Staffing measures (P<.001 for all variables in all models), health deficiency scores (P<.001 for all variables in at least 1 model), COVID-19 hospitalization rates (P<.001 for all variables in at least 2 models), and vaccinations (P<.001 for all variables in at least 2 models) exhibited meaningful relationships with the COVID-19 outcomes, while the 5-star components, Medicaid dependency, and ownership showed no clear relationships.

Although widely used, the 5-Star Quality Rating System for nursing homes is an unreliable performance measure. Concerted efforts from lawmakers, policy makers, and lobbyists are needed to refine and enhance the measure, thereby ensuring its reliability and effectiveness.

Figures