Fuzzy Approach for Audio-Video Emotion Recognition in Computer Games for Children

TL;DR

This paper introduces a fuzzy-based framework for recognizing and analyzing children's emotions during gameplay using audio-visual data, aiming to improve game design and parental oversight.

Contribution

It presents a novel fuzzy inference system for integrating audio and video emotion recognition in children's computer games, including emotion stability and diversity analysis.

Findings

Promising preliminary results on three different game types.

Effective detection of facial and audio emotions during gameplay.

Potential for enhancing child-oriented game development.

Abstract

Computer games are widespread nowadays and enjoyed by people of all ages. But when it comes to kids, playing these games can be more than just fun, it is a way for them to develop important skills and build emotional intelligence. Facial expressions and sounds that kids produce during gameplay reflect their feelings, thoughts, and moods. In this paper, we propose a novel framework that integrates a fuzzy approach for the recognition of emotions through the analysis of audio and video data. Our focus lies within the specific context of computer games tailored for children, aiming to enhance their overall user experience. We use the FER dataset to detect facial emotions in video frames recorded from the screen during the game. For the audio emotion recognition of sounds a kid produces during the game, we use CREMA-D, TESS, RAVDESS, and Savee datasets. Next, a fuzzy inference system is used for the fusion of results. Besides this, our system can detect emotion stability and emotion diversity during gameplay, which, together with prevailing emotion report, can serve as valuable information for parents worrying about the effect of certain games on their kids. The proposed approach has shown promising results in the preliminary experiments we conducted, involving 3 different video games, namely fighting, racing, and logic games, and providing emotion-tracking results for kids in each game. Our study can contribute to the advancement of child-oriented game development, which is not only engaging but also accounts for children's cognitive and emotional states.