The color of smiling: computational synaesthesia of facial expressions

TL;DR

This paper explores how facial expressions can be translated into color responses using an information-theoretic approach, proposing a latent affective space as a mediator for natural interaction and affective computing.

Contribution

It introduces an information-theoretic framework for transcoding facial expressions into color stimuli, emphasizing the role of a latent affective space as a mediator.

Findings

Affective lamp responds to facial expressions with specific colors.

The information bottleneck principle justifies the latent affective space.

Framework supports natural interaction in affective computing.

Abstract

This note gives a preliminary account of the transcoding or rechanneling problem between different stimuli as it is of interest for the natural interaction or affective computing fields. By the consideration of a simple example, namely the color response of an affective lamp to a sensed facial expression, we frame the problem within an information- theoretic perspective. A full justification in terms of the Information Bottleneck principle promotes a latent affective space, hitherto surmised as an appealing and intuitive solution, as a suitable mediator between the different stimuli.