Trends in the Japanese National Medical Licensing Examination: Cross-Sectional Study

TL;DR

The Japanese medical licensing exam has shifted over 20 years to emphasize practical problem-solving and real-world clinical skills over memorization.

Contribution

First comprehensive analysis of how Japan's medical licensing exam evolved using rule-based classification and topic modeling of 11,540 exam items.

Findings

Exam items on common/emergent diseases increased from 60% to 76% (P<.001)

Items assessing emergency care knowledge rose from 21% to 29% (P<.001)

Items requiring advanced analytical skills increased from 4% to 19% (P<.001)

Abstract

The Japanese National Medical Licensing Examination (NMLE) is mandatory for all medical graduates seeking to become licensed physicians in Japan. Given the cultural emphasis on summative assessment, the NMLE has had a significant impact on Japanese medical education. Although the NMLE Content Guidelines have been revised approximately every five years over the last 2 decades, objective literature analyzing how the examination itself has evolved is absent.

To provide a holistic view of the trends of the actual examination over time, this study used a combined rule-based and data-driven approach. We primarily focused on classifying the items according to the perspectives outlined in the NMLE Content Guidelines, complementing this approach with a natural language processing technique called topic modeling to identify latent topics.

We collected publicly available NMLE data for 2001-2024. Six examination iterations (2880 items) were manually classified from 3 perspectives (level, content, and taxonomy) based on pre-established rules derived from the guidelines. Temporal trends within each classification were evaluated using the Cochran-Armitage test. Additionally, we conducted topic modeling for all 24 examination iterations (11,540 items) using the bidirectional encoder representations from transformers–based topic modeling framework. Temporal trends were traced using linear regression models of topic frequencies to identify topics growing in prominence.

In the level classification, the proportion of items addressing common or emergent diseases increased from 60% (115/193) to 76% (111/147; P<.001). In the content classification, the proportion of items assessing knowledge of pathophysiology decreased from 52% (237/459) to 33% (98/293; P<.001), whereas the proportion assessing practical knowledge of primary emergency care increased from 21% (95/459) to 29% (84/293; P<.001). In the taxonomy classification, the proportion of items that could be answered solely through simple recall of knowledge decreased from 51% (279/550) to 30% (118/400; P<.001), while the proportion assessing advanced analytical skills, such as interpreting and evaluating the meaning of each answer choice according to the given context, increased from 4% (21/550) to 19% (75/400; P<.001). Topic modeling identified 25 distinct topics, of which 10 exhibited an increasing trend. Non–organ-specific topics with notable increases included “comprehensive clinical items,” “accountability in medical practice and patients’ rights,” “care, daily living support, and community health care,” and “infection control and safety management in basic clinical procedures.”

This study identified significant shifts in the Japanese NMLE over the past 2 decades, suggesting that Japanese undergraduate medical education is evolving to place greater importance on practical problem-solving abilities than on rote memorization. This study also identified latent topics that showed increased prominence, possibly reflecting underlying social conditions.

Figures