Patient experience with hospital care following the Maryland global budget revenue model: A difference-in-difference analysis

TL;DR

This study examines how a healthcare payment model in Maryland affected patient experiences, finding both positive and negative changes in hospital care ratings.

Contribution

The study provides new evidence on the impact of the Maryland Global Budget Revenue model on patient experience using a difference-in-difference analysis.

Findings

GBR hospitals saw significant declines in patient willingness to recommend the hospital and 9–10 ratings.

GBR hospitals showed significant improvements in cleanliness ratings.

No significant changes were observed in communication or pain control domains.

Abstract

As a result of the success of Maryland’s full risk capitated payment model experiment (Global Budget Revenue) in constraining healthcare costs, there is momentum for expanding the reach of such models. However, as these models are implemented, studies analyzing their long-term effects suggest unintended spillover effects that may ultimately influence patient experiences. The aim of this study was to determine whether implementation of the GBR was associated with changes in patient experience.

Cross-sectional study using a difference-in-difference analysis to examine changes in patient experiences according to the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) domains before and after implementation of the GBR model. Acute care hospitals from 2010–2016 with completed HCAHPS surveys were included. Hospitals identified for inclusion were then matched, based on county location, to area level characteristics using the Area Health Resource File.

A total of 844 hospitals were included. Compared to hospitals in non-GBR states, hospitals in GBR states experienced significant declines in the following HCAHPS domains: “would definitely recommend the hospital to others” [Average treatment effect (ATT) = -1.19, 95% CI = -1.97, -0.41)] and 9–10 rating of the hospital (ATT = -0.93, 95% CI = -1.71, -0.15). Results also showed significant increases in the HCAHPS domains: “if patient’s rooms and bathroom were always kept clean” (ATT = 1.10, 95% CI = 0.20, 2.00). There were no significant differences in changes for the other domains, including no improvements in: nursing communication, doctor communication, help from hospital staff, pain control, communication on medicines, discharge information, and quietness of the patient environment.

These findings suggest there should be efforts made to ascertain and mitigate potential adverse effects of care transformation initiatives on patient experience. Patients are stakeholders and their inputs should be sought and incorporated in care transformation efforts to ensure that these models align with improved patient experiences.

Figures

Average treatment effect (ATT) of patient-reported experiences according to measured HCAHPS domains for GBR implemented and non-implemented states.

Full text

Introduction

The state of Maryland’s full risk capitated payment model experiment (Global Budget Revenue) which ran from 2014–2019 demonstrated success in constraining healthcare costs to the tune of almost 1 billion dollars [1, 2]. Additionally, there were improvements in objective patient outcomes (i.e., complications, in-hospital admissions, length of stay, mortality) through the model’s running period [3–9]. Although the Centers for Medicare and Medicaid Services (CMS) through the Center for Medicare and Medicaid Innovation (CMMI) has overseen experiments with other payment reform models (e.g., bundled payments), based on the success of Maryland’s GBR, CMS has signaled a commitment to accelerating the adoption of full risk capitated models. Examples demonstrating this commitment include Maryland’s Total Cost of Care Model (TCOC), Pennsylvania’s Rural Hospital Model and the newly proposed All-Payer Health Equity Approaches and Development (AHEAD) model, inviting states to adopt statewide capitated models [10–12].

Inherent in operationalizing the mandates of Maryland’s GBR was the necessity to redesign elements of care delivery which evidence shows resulted in second order effects such as centralization of complex surgical procedures and increased volume of outpatient services outside of hospital-based domains [13]. While the goal of administrators operating under various healthcare reform mandates may be assumed to be geared toward improved patient experience, the effects of their reform programs on patient experience, intended and unintended, need to be tested.

Implementation of Maryland’s GBR achieved reductions in hospital expenditures and improvements in objective measures of patient care; however, its effect(s) on patient experience, a central tenet of patient-centered orientation to healthcare delivery, remains largely unknown. The aim of this study was to evaluate the influence of Maryland’s GBR on several domains of patient’s experience of inpatient care (a direct target of the GBR in cost containment and quality terms). Patient hospital ratings are factored into the size of global budget adjustments, incenting hospitals to focus on patient experiences to improve their ratings. Thus, we hypothesized that implementation of this all-payer model would be associated with improvements in measured patient experiences with care.

Study data and methods

Data samples

We used the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey from the CMS Hospital Compare database to examine patient reported experiences with inpatient care. HCAHPS is a standardized, publicly reported survey of patient perspectives of hospital care administered to a random sample of adult patients across medical conditions between 48 hours and six weeks after hospital discharge [14]. The survey evaluates patient experience across several domains including: communication with nurses and doctors, responsiveness of hospital staff, cleanliness and quietness of the hospital environment, communication about medication, discharge information, overall rating of the hospital, and if they would recommend the hospital to others. Survey response options are “never,” “sometimes,” “usually,” or “always” for all measures except for discharge information (yes/no), hospital recommendation (“definitely no,” “probably no,” “probably yes,” or “definitely yes”), and overall hospital rating (0, worst possible hospital to 10, best possible hospital). Surveys are administered through mail, email, or telephone and results are publicly available each year. Hospitals with completed HCAHPS surveys between 2010–2016 were included in this study. The full HCAHPS survey is publicly available at https://www.hcahpsonline.org/en/survey-instruments/.

Hospitals identified for inclusion from the HCAHPS database were then matched, based on county location, to area level characteristics using the Area Health Resource File (AHRF). This is an annual compilation of more than 6,000 variables collected from more than 50 databases to provide comprehensive county level information for health facilities. We used the AHRF to identify county-level health related characteristics that may influence patient-facing hospital operations including: county level number of inpatient days, number of Medicaid days, number of Medicare days, population percentage of those greater than 65 years of age, percentage of full-time registered nurses (FTE RNs), rate of preventable hospital stays, and percentage of those in poverty [15].

Exclusion criteria

States began adoption of the Caregiver Advise, Record, Enable (CARE) act in 2014. The CARE act required hospitals to notify patients of the opportunity to designate a caregiver while also engaging and enabling the caregiver in a patient’s care by providing education on any home healthcare tasks [16]. Recent evidence suggests that passage of this act may have resulted in improved patient experiences (measured by the HCAHPS instrument) and would confound examination of the effect of the GBR [16]. The state of Maryland adopted the CARE act in 2016, hence, we excluded states that adopted the CARE act prior to 2016 from analysis [16]. Other exclusion criteria include the United States territories and states that have not expanded Medicaid or did not expand Medicaid at the time of Maryland’s Medicaid expansion in January 2014. A summary of states chosen for inclusion in the analysis based on the status of CARE passage and Medicaid expansion is shown in S1 Table.

Exposure and statistical analysis

The primary outcome of this study was patient-reported experiences measured by the HCAHPS survey aggregated to the hospital for each year of the study (2010–2016). To examine the potential influence of Maryland’s GBR on patient experiences, we created a treatment indicator for the GBR. For hospitals in states without the GBR (i.e., all states other than Maryland) the treatment indicator variable was set equal to 0. For hospitals in Maryland, the variable was set equal to 0 before 2014, and equal to 1 from 2014–2016. All county level variables were log-transformed and normalized.

Statistical analyses include two-sided, two-sample t-tests for comparing county level number of inpatient days, number of Medicaid days, number of Medicare days, population percentage of those greater than 65 years of age, percentage of FTE RNs, rate of preventable hospital stays, and percentage of those in poverty between Maryland and Medicaid expanded non-GBR states. We deployed a fixed effects difference-in-difference (DID) analyses to assess differences in average treatment effect (ATT) of patient-reported experiences for GBR implemented and non-GBR implemented states, before and after implementation. Briefly, A difference-in-difference analysis is a statistical technique used to compare the changes in outcomes over time between a group that experienced an intervention and a group that did not, helping to isolate the effect of the intervention. The outcome variable for each patient-reported experience domain was defined as the percentage of “always” for nursing communication, doctor communication, help from hospital staff, pain controlled, medication communication, quietness of patient environment, and cleanliness of patient environment. Additionally, the experience domain was defined as the percentage of “yes” for discharge information, “definitely yes” for hospital recommendation, and a 9–10 rating for overall hospital rating. To test for violations of the parallel trends assumption, we trended the unadjusted responses across the study period (S2 Table) and used event study regression to assess that the average best-case response over time (S1 Fig) for the intervention group and the control group were parallel before the policy intervention. DID analyses included both univariable models and multivariable models with previously outlined county level covariates controlled by inverse probability weighting estimation. This study utilized publicly available and de-identified hospital-level and county-level data and was therefore exempt from institutional review board approval.

Study results

Cohort characteristics

From 2010 to 2016, a total of 11 states met the inclusion criteria (Medicaid expansion and adoption of the CARE act after 2016 or non-adoption of the CARE act) resulting in 864 hospitals eligible for inclusion in the analysis. We excluded an additional 20 hospitals (2.3% of the sample) with 0 inpatient days and stay rates of 0%. These 20 hospitals were excluded to account for noise from centers that do not have the resources to admit patients. Further, the smallest non-zero number of inpatient days and stays rates recorded for these 864 hospitals were 110 and 564, respectively. The final analytical sample of 844 hospitals includes 43 hospitals with exposure to the GBR and 801 hospitals without exposure. Hospital characteristics are displayed in Table 1. Overall, hospitals in Medicaid with exposure to the GBR had more inpatient days (t = 6.7, p-value < 0.05), more Medicaid days (t = 6.5, p-value < 0.05), more Medicare days (t = 6.0, p-value < 0.05), a larger FTE RN percentage (t = 5.7, p-value < 0.05), and a lower poverty percentage (t = -2.8, p-value < 0.05) across the seven year study period.

Table 1: Characteristics of hospitals according to GBR program status, including all years, inpatient days > 40, stays rate > 0, and non-missing data.

Difference-In-difference comparison of patient experiences

Average treatment effect in HCAHPS domains scores in GBR and non-GBR hospitals are shown in Fig 1 and Table 2. The parallel trends assumption was not violated for any HCAHPS domains. There were significant differences in HCAHPS score changes across several domains of patient experience. Compared to hospitals in non-GBR states, hospitals in GBR states experienced significant declines in the following HCAHPS domains: “would definitely recommend the hospital to others” (ATT = -1.19, 95% CI = -1.97, -0.41) and 9–10 rating of the hospital (ATT = -0.93, 95% CI = -1.71, -0.15). Results also showed significant increases in the HCAHPS domains: “if patient’s rooms and bathroom were always kept clean” (ATT = 1.10, 95% CI = 0.20, 2.00). There were no significant differences in changes for the other domains, including no improvements in: nursing communication, doctor communication, help from hospital staff, pain control, communication on medicines, discharge information, and quietness of the patient environment. The unadjusted mean HCAHPS scores according to GBR program status over the entire study period and pre/post-2014 are shown in Table 3.

Average treatment effect (ATT) of patient-reported experiences according to measured HCAHPS domains for GBR implemented and non-implemented states.

Table 2: Group-time average treatment effect (ATT) in measured patient experiences according to HCAHPS domains between GBR and non-GBR program hospitals, unadjusted and adjusted with covariates.

Table 3: Characteristics of HCAHPS unadjusted mean score according to GBR program status over entire study period and pre/post-2014.

Discussion

In this study examining the effect of Maryland’s GBR on patient experiences with inpatient care, we found that the GBR program was associated with a significant increase in the HCAHPS domains “if patient’s rooms and bathroom were always kept clean”, but also significant decreases in some domains of patient experience. Following adoption of the GBR, a lower percentage of patients discharged from hospitals operating under the GBR would recommend the hospitals to family and friends and a lower percentage of patients would give the hospitals the highest rating compared to patients in non-GBR states. These findings are pertinent in the context of accelerating trends in healthcare delivery and payment reform efforts because they show potential “spill-over” effects that may adversely impact important planks of delivery reform like patient-centeredness.

Evaluation of the Comprehensive Care for Joint Replacement program demonstrated lowered expenditures largely from less discharges to post-acute care facilities with spillover effects of similar size and magnitude in post-acute care related expenditures in non-targeted populations [17]. Offodile et al. in their recent study, theorized that implementation of the GBR model may incent hospitals in aggregate to allocate resources to cost-efficient and high-quality services resulting in an unintended centralization of care for complex patients. They demonstrated that following implementation of the GBR model, some complex surgical procedures including gastrectomy, pneumonectomy/lobectomy, and hip/knee revision procedures were increasingly performed at high-concentration hospitals in Maryland [13]. The effect of this centralization of complex services on access and ancillary burden on patients and caregivers, which can influence care experiences is unknown. Therefore, with the acknowledgement that full risk capitated models have so far been superior in achieving cost-containment goals compared to other payment reforms [18, 19], we must heed the cautionary dictum that whole population oriented capitated models like Maryland’s GBR are effective but may also be blunt instruments for which additional considerations must be taken to mitigate adverse unintended consequences.

It is unclear from this study why there were declines in the percentages of top hospital ratings and recommendation of hospitals to loved ones. Other study designs and data sources would shed light on this question; however, our study results are not isolated in their substance. Recent external evaluations of the GBR and subsequent TCOC model, also found that Maryland hospitals were rated lower than comparison groups on nearly all measures of patient experience, including patient hospital ratings, despite a reported focus by hospital leaders to improve patient experiences [1, 2, 12]. Together, these findings demonstrate the need for focused attention on the broader impact that redesign strategies have on patient’s experiences with care. Although data from the HCAHPS survey do surmise patient sentiment on broad themes of care experience, they lack granularity to offer specific insights for problem solving and lack in timeliness for prompt resolution as there is a lag between episodes of care, completion of surveys, and availability of results. These factors may contribute to putting providers on the backfoot in efforts to improve patient ratings of their care experience. A recent systematic review examining strategies to improve hospital care satisfaction based on HCAHPS domains concluded that the peer-reviewed record of interventions lacks comprehensiveness and leaves administrators with little more than anecdotes to guide strategic decision making [20]. Grievance and complaint resolution mechanisms are statutorily mandated as a component of Medicare Conditions of Participation. Arguably, these mechanisms can provide more granular and timely details about issues that influence ratings as they originate expressively from patients, families, and caregivers proximate to the episode of care in question. Additionally, given the free form and open nature of the process, details regarding wider domains of care experience than that provided by HCAHPS can be garnered. Grievance and complaint mechanisms should remain a plank of assessing patient experience with care delivery especially with efforts researchers have taken to taxonomize complaints for more efficacious resolution [21, 22]. However, with a specific focus on unintended adverse effects of planned care transformation initiatives, preemptive efforts should be taken to herd off potential adverse effects.

Although factoring patient hospital ratings into the size of global budget adjustments, as Maryland’s model does, incents hospitals to focus on patient experiences to improve their ratings, this focus should come with insights into pertinent patient needs. An example of this can be found in another statewide policy initiative which offers a precedent for how pertinent patient needs are considered in shaping reform priorities. The state of Oregon’s Coordinated Care Organizations (CCOs), established by the state’s legislature following passage of the Affordable Care Act, are local networks of providers who provide care to Medicaid enrollees under a capitated funding mechanism. By law, CCOs are required to have at least one Community Advisory Council (CAC) that among other functions help guide organizational priorities with health needs assessments and community specific plans for impactful services and improvements in care delivery. Although, to date, there has not been an assessment of the impact of Oregon’s CACs on the overarching objectives of cost containment and improvement of healthcare access and quality nor on patient experience of care which is the focus of this study, on the face of it, there is merit to provider engagement with communities on reform initiatives which may in intended and unintended ways impact their experience with care. The CARE act is an example of improved experience of care through response to community priorities in care delivery. The CARE act is the product of advocacy on behalf of the community served by AARP that explicitly puts on providers the burden of communication with designated caregivers to enhance care transitions following hospitalization. In a comprehensive comparison of hospitals in states that adopted the CARE act to those in states that did not, researchers found differential improvements in patient ratings for nurse and physician communication, provision of discharge information, and hospital ratings in hospitals under the CARE act mandate [16]. With emphasis on collaboration with caregivers during the discharge process, patients reported improved preparedness for discharge and improved communication resulting in measurable improvements in patient care experience [16]. So far, the CARE act has been adopted in 42 states including the state of Maryland in 2016. Hence, it warrants examination if adoption of the patient-centered CARE act in Maryland reverses the negative trend in patient ratings that this study found following GBR implementation.

Study limitations

There are several limitations that should be noted in this study. The focus of this study was to analyze patient experiences with care according to measured HCAHPS domains; however, these measures are not entirely representative of the quality of a patient’s care and their experiences with care and ultimately limits the generalizability of the findings of this study. Despite this, HCAHPS scores are a valid assessment of hospital quality, and a metric that is used in quality reporting and value-based purchasing programs [12]. As this was an observational study, there may be unobserved factors responsible for the differential changes seen, despite attempting to adequately control for competing factors and hospital characteristics. Furthermore, the hospital overall star rating was not implemented until July 2016, and we were unable to incorporate this measure in our analysis. Lastly, our analysis terminated at the end of 2016 due to the implementation of the CARE program. Therefore, the findings of our study may reflect early effects and experiences of the GBR model.

Conclusion

The results of this study suggest that implementation of a full risk capitated hospital funding model in the state of Maryland was associated with differential decreases in HCAHPS patient experience scores, specifically with decreases in overall hospital rating and recommending the hospital to others, compared to hospitals in states without similar funding mechanisms. These findings suggest there should be efforts made to ascertain and mitigate potential adverse effects of care transformation initiatives on patient experience. Patients are stakeholders and their inputs should be sought and incorporated in care transformation efforts to ensure that these cost-saving and outcomes improving models align with improved patient experiences.

Supporting information

S1 FigAverage best-case response percentage over time by intervention groups.Solid line represents a simple linear regression before and after implementation. Grey shading represents 95% CI.(DOCX)

S1 TableShowing a panel of figures according to the measured HCHAPS domains and differential change in cases vs. controls.(DOCX)

S2 TableEvent study regression results in measured patient experiences according to HCHAPS domains between GBR and non-GBR program hospitals across all years.(DOCX)