Modeling Cognitive-Affective Processes with Appraisal and Reinforcement Learning

TL;DR

This paper introduces a computational model combining reinforcement learning and appraisal theory to simulate emotional responses based on cognitive evaluations, applicable across various tasks modeled as Markov decision processes.

Contribution

The novel integration of appraisal theory with reinforcement learning formalizes emotion modeling within a task-independent framework using temporal difference updates.

Findings

Successfully predicts human emotional responses in vignette studies

Demonstrates the model's applicability to any task represented as an MDP

Highlights the role of reward processing in affective experiences

Abstract

Computational models can advance affective science by shedding light onto the interplay between cognition and emotion from an information processing point of view. We propose a computational model of emotion that integrates reinforcement learning (RL) and appraisal theory, establishing a formal relationship between reward processing, goal-directed task learning, cognitive appraisal and emotional experiences. The model achieves this by formalizing evaluative checks from the component process model (CPM) in terms of temporal difference learning updates. We formalized novelty, goal relevance, goal conduciveness, and power. The formalization is task independent and can be applied to any task that can be represented as a Markov decision problem (MDP) and solved using RL. We investigated to what extent CPM-RL enables simulation of emotional responses cased by interactive task events. We evaluate the model by predicting a range of human emotions based on a series of vignette studies, highlighting its potential in improving our understanding of the role of reward processing in affective experiences.