Implementation and evaluation of a communication coaching program: a CFIR-Informed qualitative analysis mapped onto a logic model

TL;DR

This paper evaluates a communication coaching program for medical trainees using a logic model and implementation research framework to assess its effectiveness and guide future improvements.

Contribution

The study introduces a novel approach by mapping CFIR-informed themes onto a logic model for program evaluation in medical education.

Findings

Eight themes were mapped onto four domains of a logic model: Context, Inputs/Outputs, Outcomes, and Evaluation.

Stakeholders emphasized the need for communication training and the value of the program's investment and outcomes.

Challenges in defining success and evaluating the program were identified as key areas for improvement.

Abstract

Coaching programs in graduate medical education have the potential to impact trainee development across multiple core competencies but require rigorous program evaluation to ensure effectiveness. We sought to qualitatively evaluate the implementation of a multi-departmental, faculty-led communication coaching program using a logic model framework.

Study participants were selected from four key stakeholder groups: resident coachees, faculty coaches, medical education leaders, and programmatic sponsors. 30–45 min semi-structured interviews were conducted via Zoom, transcribed, and de-identified for the analysis. Interviews captured stakeholders' perspectives on physicians' communication training needs, stakeholders perceived and actual roles, stakeholders’ involvement in the program, factors influencing the implementation process, and strategies for programmatic improvement, sustainment, and spread. The Consolidated Framework of Implementation Research (CFIR) guided the codebook development and data analysis. A combined inductive/deductive approach was used to develop a 20-item codebook, followed by a team-based thematic analysis. A strong intercoder agreement (Cohen’s kappa coefficient κ = 0.83) ensured coding consistency. The emerging themes were then mapped onto four domains of a logic model: Context, Inputs and Outputs, Outcomes, and Evaluation.

35 interviews were conducted between November 2021 and April 2022 with representation from all stakeholder groups, including 10 resident coachees (who received coaching), 10 faculty coaches (who served as coaches and underwent coaching-specific faculty development), 9 medical education leaders (who designed and implemented program), and programmatic sponsors (who provided financial support). We mapped 8 emergent themes onto the critical domains of a logic model for program evaluation. For the domain of Context, themes included (1) gap in communication education and (2) patient-centeredness. For the domain of Inputs/Outputs, themes included (1) investment in the program and (2) perceived program value. For the domain of Outcomes, themes included (1) learning-focused outcomes and (2) patient-related outcomes. For the domain of Evaluation, themes included (1) defining success and (2) challenges with evaluation.

Mapping CFIR-informed themes onto a logic model for program evaluation presents a novel strategy for integrating program implementation and evaluation, both of which are essential to effective educational programming. These findings can be used to guide future programmatic modifications to better meet the needs of key stakeholders.

Figures