Assessing Individual Competency Differences Between Third- and Fourth-Year Medical Students Using the National Clinical Assessment Tool for Medical Students in the Emergency Department

TL;DR

The study found that fourth-year medical students perform better than third-year students in emergency recognition and communication using a standard assessment tool.

Contribution

The study demonstrates that the NCAT-EM can detect competency differences between MS3s and MS4s in specific domains.

Findings

MS4s showed significantly higher performance in emergency recognition and management compared to MS3s.

MS4s also outperformed MS3s in communication skills, though not in other domains.

Trends suggested better performance by MS4s in four other domains, but these were not statistically significant.

Abstract

Medical students rotating through emergency departments as part of their clinical education are typically evaluated using an on-shift evaluation tool. The National Clinical Assessment Tool for Medical Students in the Emergency Department (NCAT-EM) is the current standard of evaluation for medical students in the emergency department, regardless of level of training. This study aims to evaluate whether the NCAT-EM can detect differences in skill levels between third-year medical students (MS3s) and fourth-year medical students (MS4s) rotating at a level 1 trauma center and teaching institution. These authors hypothesized that MS4s should outperform MS3s across all assessment domains given their additional training. A total of 930 performance evaluations were gathered for MS3 and MS4 rotating between May 2022 and June 2023. There were 321 evaluations of MS3s and 609 evaluations of MS4s. Across the six assessment domains, MS4s had statistically significant higher performances in two domains - namely emergency recognition and management (fully entrustable: 37.4% vs. 23.8% (p = 0.03)) and communication (fully entrustable: 46.2% vs. 33.6% (p = 0.03)). These findings indicate that the use of the NCAT-EM at this institution reliably differentiated between MS3s and MS4s in these two assessment domains. There were trends suggesting MS4s outperform MS3s in the other four domains, which did not rise to the level of statistical significance, but are consistent with prior validation studies of the NCAT.

Figures