REACT2023: the first Multi-modal Multiple Appropriate Facial Reaction Generation Challenge

TL;DR

REACT2023 is the first challenge to evaluate and benchmark multi-modal techniques for generating human-appropriate facial reactions in dyadic interactions, promoting collaboration across affective computing fields.

Contribution

It introduces a novel benchmark dataset and evaluation framework for multi-modal facial reaction generation, along with baseline systems and guidelines for future research.

Findings

Baseline systems provide a reference point for future improvements.

The dataset enables evaluation of spontaneous dyadic interaction scenarios.

The challenge fosters cross-disciplinary collaboration in affective computing.

Abstract

The Multi-modal Multiple Appropriate Facial Reaction Generation Challenge (REACT2023) is the first competition event focused on evaluating multimedia processing and machine learning techniques for generating human-appropriate facial reactions in various dyadic interaction scenarios, with all participants competing strictly under the same conditions. The goal of the challenge is to provide the first benchmark test set for multi-modal information processing and to foster collaboration among the audio, visual, and audio-visual affective computing communities, to compare the relative merits of the approaches to automatic appropriate facial reaction generation under different spontaneous dyadic interaction conditions. This paper presents: (i) novelties, contributions and guidelines of the REACT2023 challenge; (ii) the dataset utilized in the challenge; and (iii) the performance of baseline systems on the two proposed sub-challenges: Offline Multiple Appropriate Facial Reaction Generation and Online Multiple Appropriate Facial Reaction Generation, respectively. The challenge baseline code is publicly available at \url{https://github.com/reactmultimodalchallenge/baseline_react2023}.