A Review of Factors Influencing the National Board of Medical Examiners Clinical Subject Examination (NBME) Neurology Subject Scores

TL;DR

This review identifies factors affecting medical students' scores on the NBME Neurology exam during their clerkship.

Contribution

The study systematically reviews factors influencing NBME Neurology scores and suggests strategies to improve student performance.

Findings

Scheduling and preclinical preparation significantly impact NBME Neurology exam scores.

Educational methods vary in effectiveness for improving student performance on the exam.

Understanding these factors helps enhance residency preparedness and educational outcomes.

Abstract

The National Board of Medical Examiners Clinical Subject Examination (NBME) Neurology subject exam scores of third- and fourth-year medical students are a critical objective component of their application for a neurology residency. Multiple factors, including scheduling, preclinical preparation, and educational methods, may influence a student's performance on the Neurology NBME Clinical Science Exam (Shelf exam) during their clerkship. This review was conducted following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines to identify factors that impact medical students' performance on the NBME Neurology subject exam during their clerkship. This review synthesizes existing research to identify evidence-based strategies that optimize student performance on the Neurology Shelf exams. Understanding the factors influencing NBME scores is essential for medical students and clerkship directors to enhance educational outcomes and residency preparedness. The findings of this systematic review highlight the importance of scheduling, preclinical factors, and variations in educational methods in determining students' performance on the Neurology Shelf exam.

Figures

Article selection processPRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses

Tables

Table 1. Preclinical factors and their effect on NBME neurology subject exam scoresNBME: National Board of Medical Examiners Clinical Subject Examination

Author (year)	Variable Assessed	Shelf Score Result (SD)	NBME Shelf Score Correlation, r (95% confidence interval)	P-value
Strowd et al. (2016) [6]	Goal achievers	77.7 (7.4)	Not provided	0.2
	Goal non-achievers	75.7 (7.2)
Haidar et al. (2022) [7]	Ultrasound curriculum	87.1 (6.0)	Not provided	0.212
	No ultrasound curriculum	86.4 (6.5)

Table 2. Educational methods and their effect on NBME Neurology subject exam scoresNBME: National Board of Medical Examiners Clinical Subject Examination

Author (year)	Variable Assessed	Shelf Score Result (SD)	NBME Shelf Score Correlation (r)	P-value
Albert et al., (2014) [8]	Volume of patients and correlation to NBME Neurology score	Not reported	1) r = 0.142	1) p = 0.005
Albert et al. (2016) [9]	Volume of patients and correlation to NBME Neurology score	Not reported	r = 0.290	p < 0.01
Sampat et al. (2019) [10]	1) No practice exam	1) 78.63 (7.13)	Not reported	1 vs 2: <0.01
	2) NeuroSAE practice exam	2) 80.66 (7.17)		1 vs 3: <0.001
	3) NBME practice exam	3) 84.10 (5.04)		4 vs 5: 0.59
	4) Rotation with specialist	4) 80.59 (7.36)
	5) Rotation with general neurologist	5)80.17 (6.83)
Albert et al. (2016) [9]	1) OSCE category: diagnostic evaluation	Not reported	1) r = -0.059	0.976
	2) OSCE category: therapeutic intervention		2) r = 5.4	0.000001
	3) OSCE category: supportive intervention		3) r = 2.1	0.017
Thompson Stone et al. (2016) [11]	Bedside skills evaluation (BSE) year 1	79.3 (7.7)	Not provided	p < 0.0001
	Bedside skills evaluation (BSE) year 2	78.2 (7.8)
Lukas et al. (2014) [12]	OSCE Score	Not reported	R2 = 0.131	p < 0.001

Table 3. Clerkship structures and their effect on NBME Neurology subject exam scoresNBME: National Board of Medical Examiners Clinical Subject Examination

Author (year)	Variable Assessed	Shelf Score Result (SD)	Shelf Score Correlation, r ( 95% confidence interval)	P-value
Sampat et al. (2019) [10]	1) Surgery clerkship taken before neurology	81.77 (6.91)	Not provided	1 vs 2: < 0.01
	2) Neurology clerkship taken before surgery	79.22 ± 6.98		1 vs 2: < 0.01
	3) Clerkship completed prior to neurology	3) +0.72 (.27) Per clerkship taken before neurology		3) 0.007
Monrad et al. (2018) [13]	25% shorter clerkship (4.5 weeks)	1) 86.2 (5.2)	Not provided	0.015
	Traditional length (6 weeks)	2) 87.2 (5.6)
Nackers et al. (2022) [14]	Integrated curriculum (ForWard)	77.1 (75.3-78.9)	0.52 (from -1.31 to 2.36)	p > 0.005
	Traditional curriculum	77.7 (75.7-79.6)

Full text

Introduction and background

To accurately assess medical knowledge acquired during clerkships, the National Board of Medical Examiners Clinical Subject Examination (NBME) Clinical Science exam is used to objectively evaluate students in US Liaison Committee on Medical Education-accredited medical schools [1]. The exam is an objective measurement of student medical knowledge, unlike subjective measures such as Objective Structured Clinical Examination (OSCE) results, clerkship examinations, and evaluation by preceptors during the neurology rotation. The NBME Subject Examination in Clinical Neurology, colloquially known as the Neurology Shelf exam, is given to assess student knowledge of a specific discipline, clinical neurology, at the end of the clerkship rotation [1].

Shelf exams are scored using an equated percentage system ranging from 0 to 100%, with institutional variations in their use for grading [1]. Shelf exams may be used as a final exam, extra credit, a pass-fail test, or a determinant of honors [2]. As of 2020, the Association of American Medical Colleges (AAMC) reports that 84% of medical schools have neurology as a mandatory clerkship [3]. Although shelf scores are not directly reported to residency programs, they are an integral component of the final grade the student receives for the neurology clerkship. Residency programs value honors in clinical rotations as a key factor when selecting candidates for interviews [4]. Additionally, given that the majority of US medical schools include neurology as a core rotation, strong performance on the neurology NBME exam is imperative for increasing a student’s ranking among peers at their home institution [3]. In the past, neurology directors evaluated the United States Medical Licensing Examination (USMLE) Step 1 and 2 scores, in addition to holistic aspects of the applicant, such as geographic preference and interpersonal skills, to rank residency candidates [5]. Step 1 has now become pass/fail, and hence there is more reliance on the Step 2 score in addition to the grade achieved for the specific M3 clerkship.

There is limited comprehensive research identifying the specific factors that influence medical students' performance on the Neurology Shelf exam. By addressing this knowledge gap, the review aims to provide valuable insight to clerkship directors and educators, enabling them to design and implement effective medical education programs that optimize students' learning and performance in neurology. Our findings emphasize the importance of considering the factors that both positively and negatively impact NBME Shelf scores. Given that there are innumerable factors outside of the traditional classroom that may influence a student’s shelf exam performance, this review emphasized determining what institution-related and resource availability-related factors may be implemented to strengthen a student’s Neurology Shelf exam score. By gaining insights into these factors, medical schools can make informed decisions about curriculum design, resource allocation, and support systems to enhance students' learning experiences and improve their performance on the NBME Neurology subject exam.

Review

Methods

Two authors independently conducted a comprehensive literature search in Embase, Scopus, and PubMed on November 14, 2023, adhering to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. The search strategy included the keywords "student" and "clerkship" and “neurology” and is limited to studies in English with full text available, which were published within the past 10 years. Duplicate records were removed using Rayyan reference management software (Rayyan Systems Inc., Cambridge, MA) manually, and the abstracts of the remaining records were screened for relevance by two authors independently. A third author was consulted when an agreement could not be reached to eliminate bias in the screening process. A full-text assessment was done of 336 studies, out of which 174 duplicates and 153 articles were excluded due to failure to meet inclusion criteria. The nine remaining studies were included in this systematic review. Data on the factors influencing subject exam scores were independently extracted by three authors and synthesized in a Word document. Figure 1 illustrates the article selection process, following PRISMA guidelines, that was used in this study.

Article selection processPRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses

Inclusion criteria for the review included studies that had a comparison of factors that impact student performance on the NBME Neurology subject exam. Exclusion criteria included studies that did not have full text available, did not assess any factors, did not report neurology subject exam scores or p-values, and were not specific to neurology, other systematic reviews, and papers reporting on non-American medical schools. The final list of articles included in the review consisted of nine articles, which met our inclusion criteria. The selected articles were reviewed, and the relevant data were extracted and organized into tables. The main factors pertinent to this research were identified by grouping them into broad categories, derived from recurring themes found in the data. Given the heterogeneity of study designs and outcomes, a narrative synthesis was used for data analysis.

Results

The search retrieved 336 articles: 102 from Embase, 85 from Scopus, and 149 from PubMed. After screening, 162 were excluded, and nine met the PRISMA criteria for inclusion (Figure 1). The nine articles were classified into three primary groupings based on the focal points of the factors they highlighted. The first category included articles that examined preclinical factors. The second grouping comprised articles that examined the timing of the neurology clerkship relative to other clerkships, for variables such as scheduling and duration. The final category consisted of papers examining educational method types during the clerkship. Studies addressing multiple influencing factors were included in more than one table where applicable. Finally, all the relevant factors, results, and P-values were organized into tables.

Preclinical Factors

The impact of preclinical factors, such as education, mindset, and resources, on NBME Neurology subject exam scores was determined. These articles addressed the current status of medical education and how curriculum changes may impact future academic performance (Table 1). Strowd et al. [6] found that third-year medical students performed better on their neurology clerkships by setting pre-determined neurology-specific goals through the influence of advisors and peers. The students in the goal cohort completed a baseline survey at the time of clerkship, self-generating a list of their top 3 goals for the four-week clerkship. Achievers of these goals had NBME scores 1.7 points higher than their non-achiever counterparts (95% confidence interval: 0.1-3.2, p = 0.04) who did not set predetermined goals [6]. Haidar et al. [7] studied the incorporation of a longitudinal ultrasound curriculum that focused on head and neck, cardiovascular, abdominal, musculoskeletal, and procedural ultrasound sessions during the 1.5-year preclinical curriculum divided into blocks by organ system. The organ systems were divided into cardiovascular, pulmonary, renal, gastrointestinal, hematology/oncology, endocrine/reproduction, musculoskeletal, neurology, dermatology, psychiatry, and infectious diseases blocks. There was no significant difference in preclinical neurology block, NBME Neurology Shelf exam scores (p = 0.212), and Step 1 scores for the ultrasound group compared to those without the ultrasound curriculum [7].

Educational Method

Next, the impact of various educational resources, such as case volume, OSCE exams, and outpatient rotation opportunities, on NMBE Neurology subject exam scores was examined (Table 2). Albert et al. [8] found that the volume of cases a medical student encountered was significantly correlated with higher performance on measures of specialty knowledge and clinical skill. Thereby, an increased case volume significantly improved both the neurology NBME score(r = 0.0.142, p = 0.005) and the OSCE physical examination performance scores (r = 0.136, p = 0.007) [8]. Two years later, the study was repeated, and Albert et al. [9] found that the volume of cases a medical student encountered was significantly correlated with the Neurology NBME (r = 0.290, p < 0.001), the OSCE physical examination (r = 0.236, p = 0.011), and the OSCE patient clinical note (r = 0.238, p = 0.010) scores. In addition, they examined whether the breadth of the neurology cases observed by the student classified into 13 categories (seizure, cerebrovascular disease, peripheral/neuromuscular, spinal disease/neck/back pain, central nervous system demyelinating disease/neuro-immunology, headaches, sleep disorders, neuro-oncology, movement disorders, dementia, other neurodegenerative disorders) influenced objective scores. They found that the breadth of neurology cases significantly correlated with the Neurology NBME scores (r = 0.231, p = 0.017), though a significant correlation was noted with the OSCE score [9]. Sampat et al. [10] found that having students complete a two-week outpatient rotation in which they worked with a single general neurologist, instead of multiple different subspecialists, did not significantly affect shelf examination (p = 0.59) or standardized evaluation (p = 0.34) scores. However, the study found that taking a neurology shelf pre-test correlated with overall higher standardized evaluation scores (p < 0.01) and higher shelf examination scores (p < 0.01) [10]. Thompson Stone et al. [11] reported that incorporating a formal bedside skills evaluation (BSE) for third-year students resulted in an overall positive correlation with their shelf scores of 0.35 (p < 0.001). Lukas et al. [12] found that students who performed better in their OSCE, as indicated by their OSCE scores, did better on the NBME Neurology scores (R2Δ = 0.131, p < 0.001) and received better faculty clinical scores (R2Δ = 0.078, p < 0.001.

Clerkship Structure

Finally, the relevance of the timing of a medical student's neurology clerkship, in comparison to other clerkships during their third and fourth years, was examined (Table 3). Sampat et al. found that, for every additional core clerkship that a student completed, regardless of which clerkship it was, there was an increase of 0.72 points in the Neurology NMBE subject exam score 1 [10]. Monrad et al. [13] examined the transition period between curricular changes where the clerkship education was reduced from 48 weeks to 36 weeks, the neurology clerkship was reduced from four weeks to three weeks of mixed inpatient/outpatient experiences with a reduction in didactic sessions, and the neuro-ophthalmology conference. They found that, when the neurology clerkship length was shortened by 25%, there was no significant difference in NBME subject exam scores between the cohort with the shortened clerkship and the control group [13]. Nackers et al. [14] also observed a change when they converted their curriculum from a traditional 2+2 to an integrated clinical education. The traditional neurology clerkship was a four-week block, whereas, in the new curriculum, two weeks of neurology were integrated in the “acute care module” (internal medicine, emergency medicine, neurology, and psychiatry) and then another two weeks in the “chronic and preventative care module” (family medicine, general internal medicine, neurology, and psychiatry). However, for both curricula, the neurology shelf exam was taken after completion of the full four weeks of clinical rotation. There was no statistical difference in scores between having an integrated curriculum in which students begin clinical rotations one semester earlier during their second year and the traditional 2+2 medical school educational model in which a student completes two years of preclinical education, followed by two years of clinical education, with an integration of the traditional clerkships into four blocks [14,15].

Discussion

This systematic review highlights the complex interplay of factors influencing success in neurology clinical education. While certain factors may appear intuitively significant, detailed analysis reveals complex relationships and opportunities for targeted educational improvements. This review explores three key categories: pre-clerkship preparation, clerkship structure, and educational methods.

While some findings might seem counterintuitive, they provide valuable information for optimizing educational strategies. For example, shortening the clerkship by 25% did not significantly affect score outcomes [11]. Additionally, factors related to the preclinical years, such as setting goals or participating in an ultrasound curriculum before the clerkship, did not show a significant impact on exam performance [13]. Interestingly, the timing of the neurology clerkship played a crucial role in student scoring, with prior exposure to other core clerkships positively correlating with higher scores. The timing of the neurology clerkship significantly influenced student scores, with prior exposure to other core clerkships correlating with higher performance [11]. Resource availability for medical students emerged as another important factor. Busy rotation sites, access to practice exams, and strong faculty mentorship (both specialists and general neurologists) all correlated with better performance [10].

Although factors preceding clinical rotations, such as goal setting, mindset, and resource accessibility, did not demonstrate a statistically significant influence on NBME Neurology Shelf exam scores, they could still play a crucial role in fostering interest and motivation for the field. Studies suggest that students who set neurology-specific goals and hold a positive view of the subject tend to perform better, potentially hinting at the link between early engagement and future career paths [6]. This raises intriguing questions about whether interventions designed to spark a passion for neurology during preclinical years could ultimately help address the field's growing need for specialists. Seven factors were found to influence a medical student’s interest in a specific specialty prior to rotations, including medical lifestyle, encouragement, positive clinical exposure, discouragement or negative clinical exposure, economics or politics, competence or skills, and ease of residency entry [16]. Therefore, optimizing NBME performance and clinical clerkship experience could help cultivate medical students' interest in neurology.

A variety of educational exposures significantly impacted NBME Neurology Shelf exam scores, but their ultimate effects varied. The hands-on clinical experience proved beneficial, with studies showing a weakly positive correlation between case volume and breadth and shelf exam scores [8,9]. This suggests prioritizing busier rotation sites to obtain maximal exposure of case variety. Additionally, strong performance in subjective assessments such as OSCE and faculty clinical scores translated to higher exam scores [11]. Apart from OSCE examinations, taking a neurology shelf practice exam, including both the NeuroSAE practice exam and NBME practice exam, showed statistically significant improvement in the actual NBME neurology exam [10].This highlights that offering more study resources for students, even in clinical rotation, leads to improved performance on exams. Sidhu et al. [16] found that, when students were offered a comprehensive review prior to their NBME Psychiatry Subject Exam (PSE), they performed significantly better, with the average score being 84.53 versus 77.15 for matched controls (p < 0.00010). This knowledge has the potential to be applied to neurology education to benefit student performance. Finally, there was no statistically significant difference between a rotation done with a general neurologist versus a specialized neurologist [1]. However, this does not negate some of the valuable education that students may gain from doing a rotation with a specialist neurologist, which may not result in a higher exam score.

Additionally, student differences in OSCE style, as noted by scorers, held predictive value for the Neurology NBME exam, showcasing the complex interplay of factors influencing performance [17]. OSCE styles were defined as supportive, therapeutic, and diagnostic. Afterwards, students were scored in these three different categories, and the predictive power of their score on the NBME neurology subject exam was assessed. OSCE scores only in the therapeutic and supportive OSCE categories correlated significantly with both NBME written exam scores and composite faculty clinical scores, suggesting that OSCE performance reflects not only specific neurology skills but also broader clinical competency. This relationship between OSCE style and NBME Neurology scores may also inspire clerkship directors to place more emphasis and share learning resources on the therapeutic and supportive aspects of the neurology clerkship. Overall, faculty clinical evaluations across different rotations lack consistent correlation, highlighting potential subjectivity, yet NBME standardized exams offer a reliable and objective measure of student performance. These findings support the use of OSCE during preclinical years only as a precursor to a strong performance in the neurology clerkship and suggest OSCE performance as a valuable tool for informing both student progress and educational program development. Furthermore, analyzing OSCE scores from two standardized patient encounters revealed high internal consistency and predictive validity; however, the study noted the shortcomings of this approach when it comes to external validity and generalizability, as this is a subjective testing measure [12].

Clerkship structure emerged as a key factor influencing exam performance. Completing more clerkships before neurology boosted scores [10]. Intriguingly, other studies on clerkship order in internal medicine and surgery clerkships suggest that sequence can impact performance on their respective shelf exam [10]. One potential explanation for this phenomenon may be that prior exposure to relevant knowledge from other fields, such as neuroanatomy in anatomy courses or neuro-pathology in internal medicine clerkships, might benefit students during a subsequent neurology clerkship. This prior knowledge could then serve as a foundation for building deeper neurology understanding and potentially even increasing exam scores. Additionally, clerkship structure modification, such as the implementation of an integrated curriculum (e.g., the "ForWard" model), showed no significant difference compared to traditional curricula [15]. The ForWard curriculum condenses preclinical studies into just three phases, with the second phase focusing on "types of care," instead of individual specialties. Importantly, both curricula require students to take the same board exams [14]. This finding highlights that using these integrated curricula may lead to more positive outcomes without compromising the core education of medical students. Shortening the neurology clerkship length from six to 4.5 weeks had no statistically significant impact on exam scores, suggesting that a shorter curriculum might not be detrimental to student performance in neurology and that time may be repurposed for other rotations that may involve a more expansive variety of cases [18]. It was found that a longer Internal Medicine clerkship resulted in higher test scores (coefficient = 0.23 points/week; p-value < 0.01) [18]. Thus, it may be valuable for clerkship directors to shorten the length of neurology clerkships and repurpose that time to internal medicine. Ultimately, the structure of the clerkship is crucial not only for test scores but ensuring that medical students are not compromising on any part of their education.

Despite the strengths of this systematic review, several limitations must be acknowledged. A major concern is the heterogeneity of study designs - many were retrospective, relied on institutional or self-reported data, and lacked randomization or control groups, introducing potential recall and selection biases and limiting causal interpretation. There was also significant variation in outcome measures, with studies reporting NBME performance as mean scores, percentiles, or correlations. Some focused on curriculum changes, while others examined subjective factors such as student mindset. These inconsistencies reduce the comparability and accuracy of aggregated findings. Furthermore, although prior research has explored student-patient interactions, a critical gap remains in understanding how educational environments and clerkship structures influence outcomes. Comparing inpatient and outpatient experiences across disciplines may help optimize resource use and enhance student learning.

Future research should address these limitations by incorporating a wider range of medical schools and considering additional factors potentially impacting shelf exam scores. Such comprehensive investigations can build upon our findings and refine our understanding of how to foster success in neurology education.

Conclusions

An assessment of numerous studies demonstrated that three main positive factors influenced the Neurology Shelf Exam scores of third-year medical students. Based on the findings, medical schools and educators are encouraged to implement several targeted strategies. First, increasing preclinical exposure to neurology, such as through integrated neuroscience modules, early clinical shadowing opportunities, or neurology-focused case discussions, can help cultivate interest and foundational knowledge. Second, ensuring the timely scheduling of the neurology clerkship - ideally after students have completed core rotations like internal medicine - may allow students to apply broader clinical reasoning skills to neurology cases, thereby improving performance. Third, institutions should prioritize equitable access to high-quality educational resources, including NBME-aligned practice materials, neurology review texts, and digital question banks. Faculty-led review sessions and mentorship programs can further enhance students' preparedness.

Beyond improving test scores, these interventions may elevate the overall quality of neurology education, helping foster greater student confidence, deeper clinical understanding, and potentially increased interest in neurology as a career. By aligning clerkship structure and resource distribution with evidence-based practices, medical schools can better prepare students not only for assessments but for competent, compassionate care of patients with neurological conditions. This is the first systematic review of factors influencing Neurology NBME scores, providing insights for curriculum optimization and educational quality assessment.