An Affective Situation Labeling System from Psychological Behaviors in Emotion Recognition

TL;DR

This paper introduces A-Situ, a computational framework that uses physiological behaviors and affective curves to reliably label emotions in real-world situations, bridging cognitive and affective perception.

Contribution

The novel framework models affective situations with affective curves derived from physiological data, improving emotion labeling accuracy in real-life contexts.

Findings

Affective curves effectively discriminate emotional states.

Physiological data correlates with continuous emotion labels.

System outperforms existing explicit emotion assessments.

Abstract

This paper presents a computational framework for providing affective labels to real-life situations, called A-Situ. We first define an affective situation, as a specific arrangement of affective entities relevant to emotion elicitation in a situation. Then, the affective situation is represented as a set of labels in the valence-arousal emotion space. Based on physiological behaviors in response to a situation, the proposed framework quantifies the expected emotion evoked by the interaction with a stimulus event. The accumulated result in a spatiotemporal situation is represented as a polynomial curve called the affective curve, which bridges the semantic gap between cognitive and affective perception in real-world situations. We show the efficacy of the curve for reliable emotion labeling in real-world experiments, respectively concerning 1) a comparison between the results from our system and existing explicit assessments for measuring emotion, 2) physiological distinctiveness in emotional states, and 3) physiological characteristics correlated to continuous labels. The efficiency of affective curves to discriminate emotional states is evaluated through subject-dependent classification performance using bicoherence features to represent discrete affective states in the valence-arousal space. Furthermore, electroencephalography-based statistical analysis revealed the physiological correlates of the affective curves.