Deliberate practice for retinopathy of prematurity: Retinal laser training using schematic eyes in ophthalmology education

TL;DR

A training program using deliberate practice improved ophthalmology trainees' skills and confidence in performing a laser procedure for retinopathy of prematurity.

Contribution

A multifaceted simulation based on deliberate practice was developed and shown to enhance trainee performance and confidence in a high-risk ophthalmic procedure.

Findings

Trainee rubric scores for LIO-ROP skills improved significantly from 2.94 to 3.59 out of 4.

Procedure time decreased from 17.19 to 15.14 minutes after training.

Reported confidence in performing LIO-ROP increased by 81.5%.

Abstract

Skills and confidence in performing high-risk procedures are essential for ensuring safe patient care. Deliberate practice is an instructional model designed to improve performance by engaging trainees in structured, repetitive practice with immediate feedback provided by supervisors. We developed a multifaceted simulation informed by deliberate practice and tested the hypothesis that trainee skills and reported confidence would increase after training. A multicenter prospective study was conducted at three universities in Thailand from July 1, 2023, to June 30, 2024. Sixty ophthalmology trainees participated in an introductory video for self-study and then completed a multiple-choice questionnaire to assess their baseline knowledge of laser indirect ophthalmoscopy for retinopathy of prematurity (LIO-ROP) and guide feedback by supervisors. The participants subsequently performed a simulated LIO-ROP on a schematic eye and received additional feedback based on a scoring rubric. The participants practiced on the schematic eye as much as needed to feel confident in their skills. Mean rubric scores indicative of LIO-ROP skills significantly improved from 2.94 to 3.59 out of 4 (P < 0.001), and the time required for the laser procedure decreased from 17.19 to 15.14 minutes in pre- and post-LIO-ROP practice, respectively. Rubric scores for performing the LIO-ROP on a schematic eye significantly improved across all steps of the procedure between pre- and post-LIO-ROP practice. Reported confidence in performing the LIO-ROP increased by 81.5%. Multifaceted simulated training informed by deliberate practice is a suitable instructional model for enhancing skill performance and confidence among postgraduate residents.

Figures

The study flow for participants.Abbreviations: LIO-ROP, laser indirect ophthalmoscopy for retinopathy of prematurity; VDO, Video; MCQ, multiple-choice question.

Schematic eye for LIO-ROP training and practice.(A) Model for LIO-ROP training, (B) Fundus photograph without laser reaction on carbon paper, (C) Fundus photograph with laser reaction on carbon paper, (D) LIO-ROP performed on schematic eye. Abbreviation: LIO-ROP, laser indirect ophthalmoscopy for retinopathy of prematurity.

Full text

Introduction

Skills and confidence in performing high-risk procedures are essential for safe patient care. Ideally, trainees should have sufficient time and low-risk learning opportunities to build their skills and confidence before progressing to on-the-job learning. Deliberate practice [1] is an instructional model that optimizes learning by engaging trainees in well-defined tasks at appropriate levels of difficulty under the supervision of teachers who provide formative feedback. This approach involves repetitive practice and self-monitoring, ultimately fostering mastery, enhanced skill, and improved patient safety.

Deliberate practice has been widely applied in fields such as chess, music, typing, sports, and postgraduate medical education [2,3]. For example, it has been used to teach life-saving procedures, such as cricothyroidotomy, to emergency medicine residents [4].

In pediatric ophthalmology, laser indirect ophthalmoscopy for the treatment of retinopathy of prematurity (LIO-ROP) requires a high level of knowledge and skill. Ophthalmology residents and fellows in Thailand currently receive on-the-job training at university hospitals. Postgraduate training typically begins with knowledge acquisition and progresses to hands-on practice with the patients under supervision. However, this on-the-job training is associated with substantial risks to patients, such as incorrect laser direction, complications like cataracts or macular scars, and prolonged general anesthesia due to extended procedure times. These challenges and the inherent difficulties of on-the-job training could negatively affect learning if trainees feel overwhelmed by emotional stress or lose confidence after making procedural errors [5,6].

Studies across various contexts and learner groups [7–12] demonstrate that training in simulated environments can improve knowledge, skills, and confidence in surgical procedures. While simulations would ideally be incorporated into pediatric ophthalmology training before practicing laser indirect ophthalmoscopy (LIO) on actual patients [13–17], such resources are not currently available in Thailand’s training programs. To address this gap, we developed a multifaceted deliberate practice module using an LIO-ROP simulator.

This study’s purpose was to evaluate the effectiveness of a multifaceted deliberate practice-based training module for LIO-ROP using schematic eyes and assess its impact on trainee skills and confidence.

Methods

A multicenter prospective study was conducted in the Department of Ophthalmology at three universities in Thailand from July 1, 2023, to June 30, 2024. All procedures involving human participants were conducted in accordance with the ethical standards of the Human Research Ethics Committee of the Faculty of Medicine at Prince of Songkla University (REC.66-216-2-1), Khon Kaen University (HE661350), and Chiang Mai University (OPT-2566–0330). The participants were residents and fellows from the ophthalmology departments of three universities. All participants were verbally informed by the investigators, in the absence of witnesses, before signing the consent form. They were provided with a hard copy of the consent form, which included comprehensive details of the study, as well as an emergency contact number.

Educational intervention

The multifaceted deliberate practice module comprised the self-study LIO-ROP video (VDO), multiple-choice questions (MCQ) for baseline knowledge assessment, gap knowledge feedback, pre-practice LIO-ROP using the schematic eye, skill assessment based on a rubric score and supervisor feedback, self-practice using the schematic eye, post-practice LIO-ROP on the schematic eye, and skill assessment with rubric scoring by the supervisor. A Usability Experience Questionnaire (UEQ) was used for product evaluation, along with a self-confidence assessment.

Learning scheme

Sixty participants from three universities (20 per university) provided informed consent and shared their demographic information before beginning the self-study VDO session. Baseline knowledge of the LIO-ROP steps was assessed using an MCQ test, followed by feedback and clarification of correct answers from supervisors. Pre-practice sessions on the schematic eye included formative feedback on participants’ techniques, based on rubric scores (described below). Participants then engaged in self-practice on the schematic eye until they demonstrated proficiency and felt confident enough in their own skills for the next step (Fig 1).

The study flow for participants.Abbreviations: LIO-ROP, laser indirect ophthalmoscopy for retinopathy of prematurity; VDO, Video; MCQ, multiple-choice question.

VDO and schematic eye session

An 8-minute VDO session was developed for self-learning, covering the basics and safety of LIO-ROP, treatment steps, instrument setup, laser settings, and serious complications. A portable schematic eye was created using 3D printing at a reasonable cost of approximately 130 US dollars. As shown in Fig 2, a fundus photograph of the ROP was printed and placed on carbon paper to simulate the laser treatment area.

Schematic eye for LIO-ROP training and practice.(A) Model for LIO-ROP training, (B) Fundus photograph without laser reaction on carbon paper, (C) Fundus photograph with laser reaction on carbon paper, (D) LIO-ROP performed on schematic eye. Abbreviation: LIO-ROP, laser indirect ophthalmoscopy for retinopathy of prematurity.

Learning assessment

Baseline knowledge assessment.

An 18-item MCQ was developed based on content requirements corresponding to the VDO (S1 file). Content experts verified the questions, which covered LIO safety, treatment steps, essential instruments, laser settings, and precautions for complications.

Skill assessment.

The LIO-ROP procedure was reviewed by pediatric ophthalmologists, and its steps were summarized by consensus. A scoring rubric was developed, modified from the ICO Ophthalmology Surgical Competency Assessment Rubric (ICO-OSCAR) for pan-retinal photocoagulation [18], a standardized tool for evaluating office-based laser procedures for retinal diseases (S2 file). The LIO-ROP skill assessment included 13 treatment steps and five global indices. The performance levels for each step were graded using the modified Dreyfus model as follows: 1 = novice, 2 = beginner, 3 = advanced beginner, and 4 = competent [19]. Assessments were administered before and after LIO practice to measure improvements in surgical skills.

Confidence and user experience assessment.

Participants evaluated the schematic eye model using the Thai version of the Usability Experience Questionnaire (UEQ) [20], which includes dimensions such as attractiveness, perspicuity, efficiency, dependability, stimulation, and novelty. Written feedback on the schematic eye model, training flow, and overall experience was also collected (Fig 1, S3 file).

Statistical analysis

The sample size was calculated [21,22], to detect differences in evaluation rubric scores between the pre- and post-training LIO-ROP sessions on the schematic eye. Based on a power of (beta) 0.8 and an alpha of 0.05, 32 participants were required to complete the module.

Continuous variables were analyzed and presented as means ± standard deviations (SD). Discrete variables were expressed as proportions (%) to illustrate the relative frequencies of the different categories. Paired t-tests were used to assess changes in continuous data before and after the intervention. The UEQ was used to collect detailed feedback on the schematic of the eye model. The internal consistency of the rubric scores and UEQ was evaluated using Cronbach’s alpha.

Results

Participant demographics are presented in Table 1. Of the 60 participants, 35 (58%) had prior experience with laser application in patients, and eight (13%) had prior experience with LIO in patients with ROP (Table 1).

Table 1: Demographic data.

The mean ± SD MCQ score after the LIO-ROP VDO session was 15.05 ± 1.57 out of 18. The LIO-ROP rubric scores significantly improved between pre- and post-practice in all steps, indicating increased trainee skills. The mean ± SD evaluation score for the steps of LIO improved from 3.00 ± 0.66 to 3.62 ± 0.48 (P < 0.001), while the mean ± SD evaluation score for global indices improved from 2.89 ± 0.67 to 3.57 ± 0.47 (P < 0.001). The mean time for LIO-ROP decreased from 17.19 minutes to 15.14 minutes (P = 0.055).

The rubric scores for performing LIO-ROP on a schematic eye between pre- and post-LIO-ROP practice were significantly improved in all steps (P < 0.05), particularly in the steps of laser spot placement, distribution, and adequate laser spot coverage (Table 2). The scores were not significantly different between the experienced and inexperienced groups for any step. Cronbach’s alpha for the rubric scores was 0.89 for pre-practice and 0.86 for post-practice.

Table 2: Differences in rubric scores between pre- and post- LIO-ROP practice.

Regarding user experience, as measured by the UEQ, participants rated the schematic eye above average in six dimensions: attractiveness, perspicuity, efficiency, dependability, stimulation, and novelty, compared with products in the benchmark dataset. The UEQ had a reliability coefficient of 0.9.

Self-reported confidence improved consistently from 56.5% to 81.5% after each step of the process (Table 3). Participants’ comments corroborated this increase in confidence in performing the LIO-ROP.

Table 3: Self-confidence for LIO-ROP after each step of training (score 0–4).

Discussion

Our multifaceted deliberate practice-based training module for LIO-ROP using schematic eyes showed that mean rubric scores, indicative of LIO-ROP skills, significantly improved in all steps, and the time required for the laser procedure decreased between pre- and post-LIO practice, indicating improved trainee skills. Additionally, trainees reported that their confidence in performing LIO-ROP increased by 81.5%.

Pediatric ophthalmology is a relatively small part of postgraduate medical education. However, as in many other specialties, trainees are required to learn high-risk procedures “on the job,” which can compromise patient safety. LIO remains a standard treatment for ROP and is critical for preventing blindness [23,24]. The duration of treatment and the adequacy of spot coverage are crucial to overall treatment success. Our study demonstrates that multifaceted simulated training informed by deliberate practice can considerably decrease treatment time through enhanced skill performance, particularly in the steps of laser spot placement, distribution, and adequate laser spot coverage. These findings have important implications for patient care and safety and for other specialties that rely on simulation-based training. The studies by Petrosoniak [4] and Cardoso [9] also showed that deliberate practice and surgical simulation training improve surgical skill.

Our study also showed consistent improvements in trainee confidence with every step of the LIO-ROP training. While competence is often regarded as a higher level of learning than confidence, both are important to consider. Gottleib [25] suggested that although competence receives the greatest attention in postgraduate medical education, confidence must also be addressed, as both are integral to ensuring safe and professional practice [12]. Repetitive surgical simulation training has been shown to improve confidence, which correlates with improved clinical practice [8,9].

Various rubric score models exist for assessing surgical skills, but few specifically address laser skills for ROP [18,26]. Based on a literature review and expert opinions, we developed a rubric to assess LIO-ROP skills. This rubric provided formative feedback that postgraduate doctors could use to improve their performance. Consistent with the principles of deliberate practice, experienced supervisors provided this feedback in a low-stakes simulation setting, encouraging trainees to use the assessment data for learning and improvement rather than treating it as a high-stakes summative assessment.

We acknowledge the limitations of this study, including the absence of a comparison group to better evaluate changes in LIO-ROP skills. Additionally, the laser reaction and retinal details in the schematic eyes may not fully replicate those in actual patients.

In future studies, researchers should explore the transferability of skills learned in multifaceted, deliberate practice-based simulated environments to real-world clinical practice and evaluate their impact on mastery learning.

In conclusion, multifaceted simulated training informed by deliberate practice represents a suitable model for enhancing the skill performance and confidence of postgraduate residents.

Supporting information

S1 FileThe multiple-choice questions.(DOCX)

S2 FileThe rubric for evaluating the LIO-ROP procedure.(DOCX)

S3 FileThe confidence assessment.(DOCX)