Discovering Effect Modification in an Observational Study of Surgical Mortality at Hospitals with Superior Nursing

TL;DR

This study introduces a method to empirically detect effect modification in observational data, demonstrating its application in comparing surgical mortality at hospitals with superior nursing versus conventional hospitals.

Contribution

The paper presents a sensitivity analysis approach that discovers effect modification using exploratory methods while controlling error rates, applied to a large Medicare surgical dataset.

Findings

Magnet hospitals show a mortality benefit less sensitive to unmeasured bias in serious surgeries.

Effect modification varies with surgical severity and patient conditions.

The method identifies groups where treatment effects are most robust.

Abstract

There is effect modification if the magnitude or stability of a treatment effect varies systematically with the level of an observed covariate. A larger or more stable treatment effect is typically less sensitive to bias from unmeasured covariates, so it is important to recognize effect modification when it is present. We illustrate a recent proposal for conducting a sensitivity analysis that empirically discovers effect modification by exploratory methods, but controls the family-wise error rate in discovered groups. The example concerns a study of mortality and use of the intensive care unit in 23,715 matched pairs of two Medicare patients, one of whom underwent surgery at a hospital identified for superior nursing, the other at a conventional hospital. The pairs were matched exactly for 130 four-digit ICD-9 surgical procedure codes and balanced 172 observed covariates. The pairs were then split into five groups of pairs by CART in its effort to locate effect modification. The evidence of a beneficial effect of magnet hospitals on mortality is least sensitive to unmeasured biases in a large group of patients undergoing rather serious surgical procedures, but in the absence of other life-threatening conditions, such as a comorbidity of congestive heart failure or an emergency admission leading to surgery.