# Variability in Serious Safety Event Classification among Children’s Hospitals: A Measure for Comparison?

## TL;DR

This study finds that hospitals classify serious safety events in children differently, making it hard to compare safety outcomes across hospitals.

## Contribution

The study evaluates variability in serious safety event classification among children’s hospitals using a common measurement system.

## Key findings

- Respondents showed substantial agreement on 4 of 7 vignettes for SSE classification (>80%).
- Three vignettes had only slight to moderate agreement in SSE classification (<70%).
- Variability in classification suggests caution when using SSE rates to compare hospital safety outcomes.

## Abstract

Hospitals have no standard for measuring comparative rates of serious safety events (SSE). A pediatric hospital safety collaborative has used a common definition and measurement system to classify SSE and calculate a serious safety event rate. An opportunity exists to evaluate the use of this measurement system.

A web-based survey utilizing 7 case vignettes was sent to 132 network hospitals to assess agreement in classifying the vignettes as SSEs. Respondents classified the vignettes according to the taxonomy used at their respective organizations for deviations and SSE classification.

Of the 82 respondents, 67 (82%) utilized the same SSE classification system. Respondents did not assess deviations for 2 of the 7 vignettes, which had clear deviations. Of the remaining 5 vignettes, 3 had a substantial agreement of deviation (>85%, Gwet’s AC ≥ 0.68), and 2 had fair agreement (<70%, Gwet’s AC ≤ 0.39). Four of the 7 vignettes had a substantial agreement on SSE classification (>80%; Gwet’s AC ≥ 0.80), and 3 had slight to moderate agreement (<70%, Gwet’s AC ≤ 0.78).

Results demonstrated agreement and variability in determining deviation and SSE classification in the 7 vignettes. Although the SSE methodology and metric used by participant pediatric hospitals yields generally similar review results, one must be cautious in using the SSE rate to compare patient safety outcomes across different hospitals.