Integrating formative and summative feedback in online situational judgement tests: effects of feedback design on medical students’ motivational and cognitive learning factors

TL;DR

This study explores how different types of feedback in online medical communication tests affect students' motivation and learning.

Contribution

The study introduces and evaluates various automated feedback designs in situational judgment tests for medical communication.

Findings

Task-based feedback did not consistently outperform performance-based feedback across all outcomes.

Expert explanations improved cognitive understanding and perceived feedback benefits.

Post-test feedback enhanced motivational outcomes like perceived competence and fairness.

Abstract

Assessment plays an important role in teaching and learning medical communication. Formative assessment is increasingly recognised as a crucial tool for improving learning outcomes and supporting competence development. In this study, we developed and evaluated different versions of automated formative and summative feedback in an online situational judgement test of basic medical communication competence.

We developed four versions of task-based (formative) feedback, differing in their enrichment strategies (i.e. reflection prompts or expert explanations) and the timing of the feedback (i.e. immediate feedback or post-test feedback), and one performance-based (summative) version featuring a test-score profile. In a randomised controlled trial with N = 269 medical students, we evaluated the effects of feedback design on participants’ (i) motivation and (ii) cognitive factors (i.e. cognitive load), as well as on their (iii) feedback perception (e.g. fairness and usefulness of the feedback) and (iv) perceived benefits of the feedback, all of which influence learning. We tested the hypotheses that task-based feedback would be more effective than performance-based feedback (H1) and that task-based versions would have differential effects on the outcome variables (H2).

Planned contrast analysis revealed that task-based feedback was not consistently more effective than performance-based feedback across all outcomes (H1). In line with H2, analyses of variance revealed differential effects of feedback enrichment and timing: expert explanations enhanced cognitive understanding and perceived feedback benefits, whereas post-test feedback improved motivational outcomes such as perceived competence and fairness.

The results highlight the complexity of determining an optimal feedback approach, as different implementations can have differential effects on the motivational and cognitive factors that further shape learning processes. The findings suggest that an optimal feedback approach depends on specific learning outcomes and student characteristics, highlighting the importance of careful selection of feedback strategies tailored to specific educational goals and contexts.

Figures