Using the Novel Musculoskeletal Emergency Medicine Assessment Tool: A Feasibility Study

TL;DR

A new tool called MEAT was tested to assess musculoskeletal knowledge in emergency medicine residents, showing it is feasible and effective for use.

Contribution

The study introduces and validates the Musculoskeletal Emergency Medicine Assessment Tool (MEAT) for assessing EM residents' MSK knowledge.

Findings

MEAT was successfully administered online with a 100% response rate among 21 participants.

Participants completed the 50-question MEAT in an average of 32 minutes with no reported difficulties.

Most participants had UME MSK education, but curricular content varied significantly.

Abstract

Background and objective

While musculoskeletal (MSK) disorders account for a significant number of primary care and emergency department (ED) visits, there are widely recognized shortcomings and gaps in MSK education throughout medical training. Undergraduate medical education (UME) frequently fails to impart clinically relevant MSK knowledge, while many emergency medicine (EM) residency graduates report feeling unprepared to manage MSK complaints. Existing MSK assessments are not tailored to EM and may inaccurately assess specialty-specific MSK knowledge. The novel validated Musculoskeletal Emergency Medicine Assessment Tool (MEAT) holds great promise in standardizing EM MSK knowledge assessment. This trial of feasibility was conducted to assess the viability and practicality of using MEAT to evaluate MSK knowledge among incoming resident physicians in EM programs.

Methods

This feasibility study involved 21 incoming EM resident physicians from two programs at a single institution. MEAT was administered online during orientation, and demographic data and survey metadata were collected. UME MSK education details were obtained, and MEAT scores were analyzed.

Results

Participants reported no difficulties in accessing or understanding the 50-question online MEAT, resulting in a 100% response rate. The average pretest score for all interns was 29.9, with a median of 30. Most participants had documented UME MSK education, but curricular content varied widely. The participants took an average of 32 minutes to complete the assessment.

Conclusions

MEAT demonstrated successful implementation and high response rates, suggesting a high level of feasibility. The tool can be used to assess baseline MSK knowledge and ultimately track progression during residency with the potential for evaluating educational interventions once further validation studies have been performed. Further adoption of MEAT across multiple EM residency programs will help to enhance the tool's generalizability.

Figures

Tables

Table 1. Demographic DataMSK: musculoskeletal; PGY: postgraduate year

Question	Answer
What institution are you from?	Academic campus, Community Campus
What year are you?	PGY-1, PGY-2, PGY-3
Where was your medical school located?	Midwest, Northeast, West, South
What type of medical degree do you have?	Allopathic, Osteopathic
What country did you attend medical school in?	(Specify)
Did you participate in any of the following rotations during medical school? Check all that apply	Physical Medicine and Rehabilitation, Orthopedics, Rheumatology, Sports Medicine, Other (specify)
Were musculoskeletal topics taught in your medical school's curriculum?	Yes or No
Were musculoskeletal topics taught in your medical school's curriculum during pre-clerkship?	Yes or No
Were musculoskeletal topics taught in your medical school's curriculum during clerkship?	Yes or No
What estimated percentage of your medical school curriculum was dedicated to musculoskeletal knowledge?	0-100%
What aspect of musculoskeletal knowledge did your medical school curriculum cover? Check all that apply	MSK Disease Treatment, MSK Imaging, MSK Pathology, MSK Physical Examination, Splinting/Casting, Other (specify)

Table 2. Demographic Data of the Respondents

Program and Degree Breakdown	Percentage	N (of 21)
Medical School Program Location
United States	100%	21
West	61.9%	13
Midwest	19.0%	4
Northwest	14.3%	3
South	4.8%	1
Medical Degree
Allopathic	66.7%	14
Osteopathic	33.3%	7
Residency Program
Community Campus	28.6%	6
Academic Campus	71.4%	15

Table 3. Musculoskeletal Medical School Curriculum-Related Responses*The one participant who responded “no” to MSK topics being taught in UME also reported participating in a sports medicine rotation in clerkshipMSK: musculoskeletal; UME: undergraduate medical education

UME MSK Curriculum	Percentage	N (of 21)
UME MSK Topics Taught	95.2%	20*
MSK Topics Taught in Pre-clerkship	90.5%	19
MSK Topics Taught in Clerkship	71.5%	15
MSK-Associated Rotations
Orthopedics	28.6%	6
Sports Medicine	23.8%	5
Physical Medicine and Rehabilitation	19.0%	4
Rheumatology	4.8%	1
Osteopathic Principles and Practices	9.5%	2
Wilderness Medicine	4.8%	1
Total Estimated Percentage of MSK-Dedicated Curriculum
Average	14.0%
Median	15%
MSK Knowledge Domains
MSK Imaging	76.2%	16
MSK Disease Treatment	66.7%	14
MSK Pathology	85.7%	18
MSK Physical Examination	90.4%	19
Splinting and Casting	38.1%	8
Osteopathic Principles and Practices	14.3%	3

Full text

Introduction

Musculoskeletal (MSK) disorders account for nearly 20% of primary care and emergency department (ED) visits annually ​[1]. Despite this prevalence, there are several documented lacunae in MSK education across all levels of medical training and practice ​[2-5]. Several studies have shown that undergraduate medical education (UME) often falls short in providing clinically relevant MSK knowledge, including diagnosis and management of common MSK conditions ​[2,3,6]. A survey revealed that only 54% of medical students felt their MSK education was adequate ​[7]. Furthermore, only 83% of medical schools require pre-clinical MSK education, and a mere 15% mandate an MSK clerkship ​[8]. This variability in UME MSK curricula underscores the challenge of defining essential foundational MSK education.

Moreover, emergency medicine (EM) residency graduates have reported feeling inadequately prepared to manage MSK complaints ​[9]. A national needs assessment of EM residencies in 2021 highlighted that most respondents felt their curriculum could be improved and expressed a desire to have a standardized MSK assessment specific to EM training ​[10]. Existing validated MSK examinations, such as the Freedman and Bernstein assessment (FB-MSK), were developed for medical students ​[2]​ and are potentially inadequate for EM residents due to their non-specialized nature and the subjective grading involved. Scoring on these assessments varies significantly based on the specialty of the test taker and who is grading the test ​[11-13]. Significant variability is noted in scores depending on the rotations a student participated in as well; e.g., those with an orthopedic rotation scored 60.3% compared with 45.1% for those without ​[11]. When testing residents, primary care trainees scored an average of 56.3%, EM residents 77.5%, and orthopedic residents 90%. This raises the concern that this tool may not be adequately relevant and effective for assessing MSK knowledge in non-orthopedic residents and other graduating medical students ​[3]. The MSK30 is another validated MSK assessment tool that uses multiple choice questions (MCQs) to mitigate the element of subjective grading, but it is recommended primarily for medical schools and primary care residencies ​[3]. Additionally, neither the FB-MSK nor the MSK30 have been described as being administered in an online format.

We recently developed and validated the Musculoskeletal Emergency Medicine Assessment Tool (MEAT), an MCQ-based tool focusing on MSK knowledge relevant to EM and designed for online administration and distribution ​[14]. To our knowledge, MEAT is the first assessment tool specifically validated for EM resident physicians. MCQ assessments like MEAT and MSK30 offer advantages such as immediate scoring and reduced grading subjectivity ​[2,3,14]. MEAT can be distributed through a variety of methods including e-mail, text messaging, or via a QR code. It can be formatted to be taken on a personal computer or handheld device. Although MCQ assessments such as MEAT have certain limitations, they are widely used in medical knowledge evaluations ​[3,15-17]. In addition, reports show that MCQ assessments are frequently administered online without any compliance or administration issues ​[16,17].

Using MEAT at various training stages enables EM residency programs to track the progression of EM-relevant MSK knowledge and evaluate the effectiveness of educational interventions, such as MSK rotations or workshops. However, there are no published trials of this assessment in EM residencies. In light of this, our feasibility study aims to assess incoming resident physicians at the onset of their graduate medical education (GME) using in-person electronic administration of MEAT.

Materials and methods

This feasibility study was conducted in 2022 and involved all 21 incoming resident physicians from two categorical EM programs at a single institution. One program is affiliated with a level-one academic teaching hospital and the other program is associated with a level-four community hospital.

Using the Qualtrics platform, the MEAT (see Appendices) was incorporated into a survey format and distributed to participants via an anonymous link during intern orientation. This assessment was completed before any formal GME MSK training, ensuring scores reflected baseline knowledge. To ensure the integrity and confidentiality of the data, each participant was assigned a randomized identification number (rID). This identifier was used for tracking scores and making future comparisons without compromising anonymity. Upon completion of the assessment, individual scores were automatically transmitted to the administrator. One point was assigned to each question, for a total possible score of 50.

Demographic data, details of the current GME program, UME location, degree type, and UME MSK curriculum information were collected during assessment administration (Table 1). Additionally, survey metadata, including the duration of each assessment, was captured for analysis. Summary statistics were computed using Microsoft Excel.

This project was reviewed and approved by the University of Arizona Institutional Review Board (IRB 1611010724, dated November 4, 2022). All participants provided informed consent, and data confidentiality was maintained throughout the study.

Results

The demographic data of the cohort are presented in Table 2. All 21 participants completed the assessment. When queried about their UME MSK training, participants provided insightful responses. Of those who participated in related MSK rotations, six residents reported completing rotations in orthopedics, while five had experience in sports medicine (SM), and four in physical medicine and rehabilitation (PMR). Additionally, one resident had undertaken a rotation in rheumatology. Three residents mentioned additional rotations they deemed relevant, including two in osteopathic principles and practices (OPP) and one in wilderness medicine.

Information regarding UME MSK curricula was collated (Table 3), which revealed that 20 out of 21 residents received MSK education during medical school. Among them, 19 reported MSK topics being covered during pre-clerkship, and 15 during clerkship. The estimated proportion of curriculum dedicated to MSK knowledge ranged from 0% to 35%, with a median of 15% and an average of 14.0%. Interestingly, one participant reported not receiving MSK education during medical school but had exposure to SM during their training. Another resident, who claimed no MSK education during pre-clerkship, had a PMR rotation where the MSK curriculum was covered. Among the five residents who reported no MSK education during clerkship, one had rotations in both orthopedics and SM, two in orthopedics only, while the remaining two did not specify any MSK-related rotations.

Regarding specific aspects of MSK knowledge covered in their medical school curricula, 16 residents mentioned training in MSK imaging, 14 in MSK disease treatment, 18 in MSK pathology, 19 in MSK physical examination, and eight in splinting and casting. Notably, three of seven osteopathic interns highlighted OPP as additional MSK training in their curriculum.

The 50-question MEAT was administered to all incoming resident physicians during intern boot camp with 100% compliance in the pretest. The average pretest score for all participants was 29.9, with a median of 30. On average, participants took 32 minutes to complete the assessment.

Discussion

The feasibility study of MEAT demonstrated a 100% response rate, indicative of successful implementation. Previous online MSK studies have also shown reasonable response rates, though never reaching 100% ​[18]. The high response rate was attributed to the in-person administration during orientation compared to remote administration. The absence of any reported difficulties in accessing, completing, or understanding MEAT indicates that it is feasible that participants will complete subsequent assessments. The mean and median participant scores of 60% indicated minimal variability in responses. While no passing score has yet been established for MEAT, the obtained scores are below conventional passing thresholds, which aligns with the expectation for participants who had not yet undergone any GME activities. Furthermore, the participants took an average of 32 minutes to complete the test, and hence sufficient time should be allocated for its administration. Overall, these findings support the feasibility and potential utility of MEAT as an efficient and accessible tool for assessing MSK knowledge among incoming resident physicians in EM programs.

All participants had documented UME MSK education, with a median of 15% of the total curriculum being dedicated to MSK education. This closely aligns with the estimated 20% prevalence of MSK complaints seen annually in primary care and ED visits. However, the content and clinical relevance of UME MSK education varied significantly, ranging from 0 to 35% of the total curriculum, reflecting variability in MSK knowledge domains and clinical rotations. Although the UME curriculum must fit within the general competencies framework required by accrediting bodies such as the Liaison Committee on Medical Education (LCME) ​[19]​ or the American Osteopathic Association Commission on Osteopathic College Accreditation (COCA) ​[20], there are no specific guidelines on the MSK education content or the phase of UME in which it should be taught. Without such guidance, there is no standardization. Without standardization, tests such as the FB-MSK, MSK30, and MEAT may have to remain specialty-specific rather than generalizable to any medical student.

Following the successful feasibility study, our next step involves leveraging MEAT to establish multiple benchmark data points throughout EM residency training. Through a multi-institutional study involving various EM programs, we aim to generate comprehensive benchmarks for incoming and outgoing EM resident physicians. Educators can use these benchmarks to evaluate the efficacy of their programs on a broader scale. Additionally, they can assess the impact of a specific intervention, such as an MSK rotation or workshop, on resident MSK knowledge. The availability of these data points will enable educators to implement targeted interventions tailored to individual residents or entire programs and drive continuous improvement in MSK education and patient care outcomes within the field of EM.

This feasibility study has several limitations. The small sample size, comprising 21 participants from a single institution, limits the generalizability of the findings. The small sample size also reduces the statistical power to detect significant differences and may not accurately represent the broader population of EM residents across different institutions. Additionally, the study was conducted at a single institution, which may introduce site-specific biases and limit the applicability of the results to other settings with different educational environments and resources. Furthermore, the study did not establish a passing score for the MEAT, meaning that the current scores cannot be definitively interpreted as indicating competency or deficiency in MSK knowledge. We also did not assess the residents' perceived ease of use or satisfaction with the MEAT after completion. Feedback from participants regarding the usability and relevance of the assessment could provide valuable insights for future iterations and broader implementation. As a feasibility study, the primary goal was to assess the practicality and initial response to the MEAT rather than to evaluate its effectiveness comprehensively. Future studies with larger, more diverse cohorts and multi-institutional participation are necessary to validate the MEAT and establish its efficacy in assessing and improving MSK knowledge among EM residents.

Conclusions

This study underscores the documented shortcomings in MSK knowledge within both undergraduate and graduate medical education, including EM. EM residency programs have attempted to address this gap through various educational methods; however, existing validated MSK assessments are neither specific to EM nor user-friendly in terms of administration and grading. The novel MEAT has proven feasible for administration, offering a standardized method for assessing EM-specific MSK knowledge. The successful implementation and high response rate in this study highlight MEAT's potential to identify baseline MSK knowledge and areas needing improvement. While broader validation across multiple institutions is necessary to enhance the generalizability of MEAT, it already fills a significant void in current EM MSK assessments. By identifying specific deficiencies, MEAT enables educators to implement targeted, trainee-specific, or program-wide curricular interventions. This tool is poised to become an integral part of EM residency programs, ultimately improving MSK competency and patient care outcomes in this crucial area of medicine.