Impact of case-specific learning goal on robotic-assisted surgery care delivery

TL;DR

This study shows that setting specific learning goals during robotic surgery training affects residents' autonomy and patient outcomes like surgery time and hospital stay.

Contribution

The study introduces a framework linking case-specific learning goals to surgical outcomes in robotic-assisted surgery training.

Findings

Residents' autonomy was positively correlated with learning goal selection and completion.

Partially completed learning goals led to longer surgery times and higher costs.

Fully completed learning goals showed a trend toward lower readmission rates, though not statistically significant.

Abstract

We hypothesize that resident’s intraoperative case-specific learning goal may influence resident intraoperative autonomy and robotic-assisted surgery (RAS) surgical outcomes measured by operative time (OT), length of stay (LOS), direct cost (DC), and 30-day readmission (30R).

Valid resident operative performance evaluations, which includes case-specific learning goal selection (LGS) and degree of learning goal completion (LGC) metrics, of three outpatient RAS procedures—inguinal hernia, ventral hernia, and cholecystectomy—performed by PGY2–5 residents were collected. Cases in which residents served as bedside assistants were excluded. OT, LOS, DC, and 30R for matched cases were extracted from hospital records. Descriptive statistical analysis was applied.

A total of 104 evaluations from 57 outpatient RAS cases were analyzed. Residents’ overall permitted intraoperative autonomy was positively correlated with their LGS (0.66, p < 0.0001) and LGC (0.64, p < 0.0001). Overall LGS had minimal impact on OT, LOS, and DC. However, significant differences (all p < 0.05) in surgical outcomes were observed between different LGC scores: Compared to cases with an LGC score 5 (fully completed learning goal), cases with an LGC score 3 (partially completed learning goal) showed significantly longer LOS (8.00 h > 5.87 h) and OT (98.36 min > 74.49 min) as well as higher DC ($4471.84 > $3945.23). RAS cases with an LGC score 5 had a higher probability of 30R than those with an LGC score 3 (18.03% > 1.64%), though it was not significant. LGS also showed a similar trend.

Study findings suggest the case-specific learning goal may influence resident autonomy as well as LOS, OT, DC and 30R of outpatient RAS cases. Identifying an achievable learning goal upon a resident’s competency level may enhance intraoperative training and RAS care outcomes.

Figures

A comparison of attending surgeon and resident scores on four evaluative metrics

Full text

Introduction

The use of robotic-assisted surgery (RAS) in surgical residency training is rapidly increasing [1]. Robotic inguinal hernia (RIH), robotic cholecystectomy (RC), and robotic ventral hernia (RVH) are three RAS procedures that are gradually being used to train a general surgery resident in academic medical centers and teaching hospitals. RAS—especially RAS with a dual console—provides convenience (e.g., pass control back and forward) for attending surgeons to facilitate teaching and grant resident autonomy in the operating room (OR) [2]. But multiple studies reported that guidelines for RAS training and intraoperative teaching were still needed [3, 4].

However, intraoperative teaching in an RAS case is a complex task with a high cognitive demand [5] and many distractors, potentially leading to detrimental clinical outcomes, such as prolonged surgery time length, increased 30-day complication rates and composite morbidity [6–9]. Although intraoperative teaching does not significantly compromise major patient safety, several factors (e.g., resident training level) may contribute to detrimental surgical outcomes based on literature [10]. But which intraoperative teaching element(s) can be improved to enhance RAS surgical outcomes remains unclear.

Traditionally, the case-specific learning goal has been a key element of intraoperative teaching in the OR [11]. A learning goal is typically the same as a resident’s primary learning or practice task in a surgical case. A clearly defined, achievable learning goal may improve residents’ intraoperative learning experience and autonomy [12]. Therefore, we hypothesize that a case-specific learning goal will not only influence resident intraoperative autonomy but also impact RAS surgical outcomes measured by operative time (OT), length of stay (LOS), direct cost (DC) [13], and 30-day readmission (30R). The goal of this preliminary study is to test this hypothesis. Study findings could potentially contribute to enhancing RAS surgical outcomes without sacrificing resident intraoperative training.

Methods

Setting and participants

This study was conducted in the Center for Minimally Invasive Surgery at an academic medical center. General surgery residents from postgraduate year (PGY) 2 to 5 and RAS faculty from Department of Surgery were eligible to participate in this study. All general surgery residents in our institution must complete a robotic surgery training curriculum prior to being permitted to operate as a console surgeon. The robotic training curriculum consists of three parts per the Intuitive training requirements for residents and fellows [14]: two online training modules, one simulation training module (on the Da Vinci SimNow and score at least 90%), and the case logs of five RAS cases as bedside-assistant. The Institutional Review Board (IRB) approved this study (#2021H0154).

Data collection and analysis

We collected resident operative performance evaluations with an augmented procedure-specific Surgical Entrustable Professional Activities (SEPA) instrument [15, 16], which contains a set of key evaluation metrics with a 1-to-5 Likert scale: case difficulty, the resident’s case-specific learning goal selection, the degree of learning goal completion, surgical timeout quality, procedure-specific skills (6 items), procedure step-specific guidance (6 items), general skills (4 items), overall guidance, operative planning and judgment, overall technical performance, overall team management, and prospective entrustment (i.e., future entrustment the attending surgeon would like to grant the resident in upcoming similar cases). Two case-specific learning goal metrics—learning goal selection (LGS) and degree of learning goal completion (LGC)—and two resident autonomy metrics from SEPA evaluations (Table 1) were included in current study.

Table 1. Descriptions of SEPA learning goal and resident autonomy metricsSEPA evaluation items (Input Metrics)Description and rating scale anchorsCase-specific learning goal (Learning Goal Selection Metric)To assess whether the case-specific learning goal is manageable and appropriate for the resident and the case Rating scale anchors: 1—Had a global non-specific learning goal3—Had a specific learning goal without manageable plans and/or tasks5—Had a specific learning goal which was manageable and appropriate for this resident and the caseAchieved case specific learning goal (Learning Goal Completion Metric)To assess whether the resident is able to complete the pre-identified learning goal by the end of the case Rating scale anchors: 1—Did not achieve learning goal3—Completed some of the learning goal5—Achieved 100% of the learning goalStep-specific guidance (Step-Specific Autonomy Metric)To assess the amount of guidance a resident needed from the attending surgeon in each procedure-specific critical step. Less guidance needed implies more autonomy Rating scale anchors: 1—Substantial guidance was provided/needed3—Some guidance was provided/needed5—Minimal guidance was provided/neededOverall guidance (Overall Autonomy Metric)To assess the overall amount of guidance provided to/received by this resident throughout the evaluated case Rating scale anchors: 1—Substantial guidance was provided/needed3—Some guidance was provided/needed5—Minimal guidance was provided/needed

A purposeful sampling approach was applied with following inclusion criteria: (1) RAS performed with a dual console Da Vinci® Xi™ surgical system (Intuitive Surgical Inc, Sunnyvale, CA), and (2) RIH, RVH, and RC cases only. Residents as bedside assistant cases were excluded. The operating resident and the attending surgeon were invited to complete the same SEPAs evaluation of resident performance in their eligible RAS case (RC, RIH, and RVH) within 3 days of each completed RAS operation via the online survey Qualtrics system (SAP SE, Provo, UT).

OT (i.e., cut-to-close time), DC, LOS (measured in hours), and 30R of matched RAS cases were retrospectively extracted from hospital records. Statistical analysis was performed using Pearson correlation, one-way ANOVA, and contingency table with JMP Pro (version 18; SAS Institute Inc, Cary, NC). Extreme sample comparison was applied to assess the impacts of the highest score and the lowest score on surgical outcomes. P < 0.05 is considered statistically significant. A retrospective power calculation (α = 0.05) was performed to determine the statistical power of study findings.

Results

A total of 104 SEPA evaluations from 57 RAS cases (RIH: 28, RC: 23, and RVH: 6) were included. Most RAS cases (80.7%, 46 out of 57) involved a PGY5 resident, followed by PGY2 residents (12.3%, 7 out of 57). PGY4 residents participated in three RAS cases, and PGY3 residents participated in two RAS cases. The majority of included cases (82.5%, 47 out of 57) received evaluations from both attending surgeons and residents.

Residents and attending surgeons’ scores are similar with variations within the same Likert rating scale except for the “Step-Specific Guidance” metric (Fig. 1). Residents’ overall permitted intraoperative autonomy – scores of graded “Overall Guidance”—was positively correlated with their LGS (r = 0.66, p < 0.0001) and LGC (r = 0.64, p < 0.0001). Similar positive correlations were observed in residents’ procedural step-specific autonomy with LGS (r = 0.64, p < 0.0001) and LGC (r = 0.57, p < 0.0001).

Fig. 1A comparison of attending surgeon and resident scores on four evaluative metrics

All cases had completed matched records for OT, DC, and LOS. Seven out of 57 RAS cases did not have 30R data available at the time of data extraction. The mean of LOS was 6.49 h [95%CI 5.66, 7.28], OT was 80.46 min [95%CI 74.59, 86.32], and DC was $5,762.66 [95%CI 3531.62, 4203.74]. Likewise, eight cases (16.0%, 8/50) had 30R.

Overall, LGS had minimal impact on OT, LOS, and DC. However, notable overall differences in surgical outcomes were observed among three LGC scores (Table 2)—lower LGC scores associated with longer OT (p = 0.0221) and higher DC (p = 0.0231); a similar trend was also observed in LOS, though it was not statistically significant. Extreme sample comparison showed when compared to RAS cases with the highest LGC score (5—fully completed learning goal), cases with the lowest LGC score (3 – partially completed learning goal) had, on average, a 2.13-h longer LOS (8.00 > 5.87, p = 0.0496, power = 0.43), a 23.87-min longer OT (98.36 > 74.49, p = 0.0051, power = 0.80), and a $526.25 higher DC (4471.48 > 3945.23, p = 0.0064, power = 0.79). Interestingly, RAS cases with a score of 5 in LGC tend to have the highest probability of 30R—LGC 5: LGC 4: LGC 3 = 12.36%: 3.37%: 1.12%. The extreme comparison showed when particularly compared RAS cases with an LGC score of 5 to those with an LGC score of 3, cases with LGC score of 5 had a higher probability of 30R (18.03% > 1.64%), though this was not significant.

Table 2. Impact of learning goal completion (LGC) on RAS outcome measuresLearning goal completion scoreRating scalep ValuePower (α = 0.05)345Length Of Stay (hours)8.006.795.870.20520.33Operative Time (mins)98.3681.4674.490.02210.70Direct Cost (dollars)4471.844034.833945.230.02310.69

Discussion

The use of robotic-assisted surgery (RAS) is rapidly increasing, simultaneously leading to a demand for enhancing resident robotic surgery training [17, 18] in order to secure the supply of high quality independent robotic surgeons for future patient care. To optimize the quality of resident training and the RAS care delivery, it is necessary to gain more knowledge about which intraoperative teaching/learning element(s) might influence robotic surgical care outcomes. Findings from this study suggest that the case-specific learning goal would not only influence resident intraoperative autonomy but also have an impact on LOS, OT, DC and 30R of outpatient RAS cases.

Setting an achievable case-specific learning goal that is appropriate for a given resident’s competency level and is mutually agreed upon by both the attending surgeon and the resident is crucial. This specific learning goal provides a clear roadmap toward the resident’s intraoperative learning target for a surgical case, helping to align the attending surgeon’s teaching with the resident’s learning efforts. If the attending surgeon agrees that the proposed learning goal is appropriate and manageable, they are typically willing to slow down and give the resident more autonomy to work on the learning goal or task, which is valuable for the resident’s competency development [19]. Therefore, further investigation into practice guidelines for RAS procedure-specific learning goals for residents becomes necessary.

Study findings also suggest that how well a resident completes their learning goal influences OT, DC, and LOS in the three RAS (RC, RIH, and RVH) study samples, which aligns with the literature [20–22]. If a resident cannot complete the learning goal or task selected for an RAS case, it may be because the attending surgeon or resident has inaccurately estimated the resident’s competency level and/or the case complexity, leading to the attending surgeon taking over the case. This is also supported by current data showing that LGC has a significant positive correlation with resident autonomy—“Overall Guidance” metric—(r = 0.64, p < 0.0001). Likewise, an ambitious learning goal commonly extends surgery duration, as the attending surgeon needs to slow down to teach and the resident also needs more time to safely practice skills, resulting in prolonged recovery time and increased cost [23].

Surprisingly, findings from this study suggest that RAS cases with a score of 5 in LGC tend to have the highest probability of 30R. One possible reason may be that a resident operating under high stress as the primary surgeon may make more mistakes in the OR according to the cognitive load theory [24–26]. Research shows that numerous medical errors affecting patient safety often result from complex human factors such as cognitive bias, time pressure, attention lapses, communication failure, insufficient skill, and fatigue [27–29]. Residents experience significant stress while operating as console surgeons under the supervision of an attending surgeon in RAS cases. According to the well-established Swiss Cheese Model of human error [30], an attending surgeon can help a resident fix all active errors—that is, errors made by the resident at the point of care—to ensure patient safety and operative efficiency in the OR. However, fixing all active errors may not prevent latent errors, which are literally "accidents waiting to happen" within 30 days of the operation [31]. Thus, current findings highlight the need to identify appropriate and achievable learning goals or tasks for residents based on their RAS experience, competency level, and cognitive load capacity to promote resident training and patient care simultaneously. It is also crucial to identify root causes of latent errors to optimize RAS outcomes without compromising resident intraoperative training [32].

This study has some limitations. First, it is a preliminary study with limited samples, leading to lack of strong overall statistical power. Second, we are unable to assess potential impact from patient-related factors on study findings. Third, it is a single institution study, and the local RAS patient population may influence the results. However, current study provides useful actionable information to help surgery residency programs optimize an intraoperative teaching protocol for RAS cases aiming to effectively teach a resident while maintaining the best RAS surgical outcomes. Lastly, we are unable to access and identify reasons for each patient’s 30R. Thus, we cannot exclude the possibility that the 30R is caused by other non-surgery related factors. Future study with a larger sample size is needed to further investigate the findings.

Conclusion

Findings from this preliminary study suggest that the case-specific learning goal may influence resident autonomy as well as LOS, OT, DC and 30R of outpatient RAS cases (RC, RIH, and RVH). Identifying an achievable learning goal upon a resident’s competency level may enhance intraoperative training in RAS and surgical care outcomes.