Supporting Cognitive and Emotional Empathic Writing of Students

TL;DR

This paper introduces an annotation scheme and predictive models for emotional and cognitive empathy in student peer reviews, supporting adaptive feedback and fostering empathy skills in educational settings.

Contribution

It presents a novel annotation scheme for empathy in student writing, trains models to detect empathy structures, and develops a system for providing personalized empathy feedback.

Findings

High inter-rater agreement for annotation components (α=0.79)

Moderate agreement for empathy scores (multi-π=0.41)

Positive impact on perceived empathy skill learning and feedback perception

Abstract

We present an annotation approach to capturing emotional and cognitive empathy in student-written peer reviews on business models in German. We propose an annotation scheme that allows us to model emotional and cognitive empathy scores based on three types of review components. Also, we conducted an annotation study with three annotators based on 92 student essays to evaluate our annotation scheme. The obtained inter-rater agreement of {\alpha}=0.79 for the components and the multi-{\pi}=0.41 for the empathy scores indicate that the proposed annotation scheme successfully guides annotators to a substantial to moderate agreement. Moreover, we trained predictive models to detect the annotated empathy structures and embedded them in an adaptive writing support system for students to receive individual empathy feedback independent of an instructor, time, and location. We evaluated our tool in a peer learning exercise with 58 students and found promising results for perceived empathy skill learning, perceived feedback accuracy, and intention to use. Finally, we present our freely available corpus of 500 empathy-annotated, student-written peer reviews on business models and our annotation guidelines to encourage future research on the design and development of empathy support systems.