Development of a Deep Learning–Based Feedback Model to Assist Medical Students Learning Renal Ultrasound Acquisition: Mixed Methods Study

TL;DR

A deep learning model was developed to provide feedback on renal ultrasound images for medical students, improving their learning and skill acquisition.

Contribution

A cascaded deep learning feedback model was created to classify and provide feedback on renal ultrasound images for medical students.

Findings

The model was positively received by students and encouraged self-regulated learning.

Student satisfaction with the model's usability was high, with 49% to 76% rating it 4-5 on a Likert scale.

OSCE scores improved slightly after model implementation, though not statistically significant.

Abstract

Point-of-care ultrasound training is being increasingly integrated into undergraduate medical education, leading to a substantial demand for trained faculty to provide instruction and feedback.

This study aimed to develop an adjunct tool, a deep learning–based feedback model, to facilitate student learning.

Renal ultrasound images (N=2807) were used to train a cascaded deep learning–based feedback model that classified images into three categories: optimal, suboptimal, and incorrect. Suboptimal images were further subcategorized as images with artifact, incorrect gain, and/or incorrect positioning. The model was deployed among year 5 medical students receiving bedside ultrasound training, who were invited to upload renal ultrasound images to an online platform for automated image quality grading and feedback. A mixed methods analysis was used to evaluate students’ learning experience. Focus group interviews were organized to qualitatively analyze the successes and challenges of implementation. Quantitative analysis was based on responses to a 5-point Likert scale questionnaire and performance on the objective structured clinical examination (OSCE). Objective structured clinical examination scores were compared with mean OSCE scores from the 2 years preceding implementation of the deep learning–based feedback model.

Focus group interviews identified that the deep learning–based feedback model encouraged self-regulated learning but also recognized that discordant curricular design and hardware limitations impeded its use. The 11-item online questionnaire had a response rate of 42.4% (98/231 students). Among respondents, 32% (31/98) to 48% (47/98) found the model helpful in assisting ultrasound training (Likert score of 4‐5 for items 1-3), while 49% (48/98) to 76% (74/98) were satisfied with its usability and their interaction with the model (Likert score of 4‐5 for items 4-11). The mean OSCE score was 9.73 (SD 0.76) out of 10, compared with mean scores of 9.35 (SD 1.03; P=.06) and 9.45 (SD 0.97; P=.15) out of 10 in the 2 individual years preceding implementation of the model.

A cascaded deep learning–based feedback model was developed and deployed among year 5 medical students receiving bedside ultrasound training, with positive learner responses and enhanced self-regulated learning. The innovation was associated with increased student engagement and improved ultrasound skill acquisition among novice learners.

Figures