pECGreview: Assessment of a Novel Tool to Evaluate the Accuracy of Pediatric ECG Interpretation Skills

TL;DR

This study evaluates a web-based tool called pECGreview to assess how accurately pediatric cardiology trainees interpret ECGs.

Contribution

The study introduces a validated subjective scoring system for evaluating pediatric ECG interpretation accuracy.

Findings

The mean accuracy score across 1460 ECG interpretations was 76.6%.

Inter-rater agreement was strong, with correlations of 0.75 or higher.

Analyzing as few as five ECGs provided reliable assessment of trainee skills.

Abstract

The skill of interpretation of the electrocardiogram (ECG) remains poor despite existing educational initiatives. We sought to evaluate the validity of using a subjective scoring system to assess the accuracy of ECG interpretations submitted by pediatric cardiology fellows, trainees, and faculty to the Pediatric ECG Review (pECGreview), a web-based ECG interpretation training program. We conducted a retrospective, cross-sectional study of responses submitted to pECGreview. ECG interpretations were assessed independently by four individuals with a range of experience. Accuracy was assessed using a 3-point scale: 100% for generally correct interpretations, 50% for over- or underdiagnosis of minor ECG abnormalities, and 0% for over- or underdiagnosis of major ECG abnormalities. Inter-rater agreement was assessed using expanded Bland–Altman plots, Pearson correlation coefficients, and Intraclass Correlation Coefficients (ICC). 1460 ECG interpretations by 192 participants were analyzed. 107 participants interpreted at least five ECGs. The mean accuracy score was 76.6 ± 13.7%. Participants were correct in 66.1 ± 5.1%, had minor over- or underdiagnosis in 21.5 ± 4.6% and major over- or underdiagnosis in 12.3 ± 3.9% of interpretations. Validation of agreement between evaluators demonstrated limits of agreement of 11.3%. Inter-rater agreement exhibited consistent patterns (all correlations ≥ 0.75). Absolute agreement was 0.74 (95% CI 0.69–0.80), and average measures agreement was 0.92 (95% CI 0.89–0.94). Accuracy score analysis of as few as five ECG interpretations submitted to pECGreview yielded good inter-rater reliability for assessing and ranking ECG interpretation skills in pediatric cardiology fellows in training.

Figures