# MaReS (Magdeburg Reflective Writing Scoring Rubric for Feedback) – development of a feedback method for reflective writing in health professions education: A pilot study in veterinary medicine

## TL;DR

This study developed a rubric called MaReS to provide feedback on reflective writing in health professions education and tested it in veterinary medicine.

## Contribution

The novel contribution is the development and pilot testing of the MaReS rubric for reflective writing feedback in health professions education.

## Key findings

- MaReS consists of twelve items rated on three-point scales and includes a rater manual and guiding questions for students.
- Inter-rater reliability for MaReS ranged from -0.08 to 0.77, indicating variability in consistency.
- MaReS showed a strong positive correlation with the REFLECT rubric in the first run but not in the second.

## Abstract

The aim of the study was to develop a scoring rubric that provides valuable feedback to students and to gather evidence for its construct validity.

The Magdeburg Reflective Writing Feedback and Scoring Rubric (MaReS) was developed in an iterative process following a symposium on reflection by a committee of the “DACH Association for Medical Education (GMA)” in June 2016.

25 essays written by 13 veterinary students were assessed by three independent raters with MaReS and by two raters with the REFLECT rubric in two runs (13 and twelve essays). Validity evidence was gathered referring to the following of Messick’s components of construct validity: content (rubric development), response process (rater manual, rater training, rating time, students’ evaluation), internal structure (inter-rater reliability, IRR), and relationship to other variables (comparison of the rating with the REFLECT rubric and a global rating scale).

The analytic rubric comprises twelve items that are rated on three-point rating scales. The authors developed an assignment with guiding questions for students and a rater manual. Results for free marginal kappa of the items of MaReS ranged from -0.08 to 0.77 for the first set of reflective essays and from 0.13 to 0.75 for the second set. Correlations between MaReS and the REFLECT rubric were positive (first run: r=0.92 (p<0.001); second run: r=0.29 (p=0.37)).

MaReS might be a useful tool to guide students’ reflective writing and provide structured feedback in health professions education. Using more essays for a rater training and more training cycles are likely to result in higher IRRs.