A Recursive Framework for Expression Recognition: From Web Images to Deep Models to Game Dataset

TL;DR

This paper introduces a recursive framework that iteratively generates datasets, trains CNN models, and uses interactive interfaces to improve facial expression recognition in real scenes, demonstrated through new datasets and a game-based approach.

Contribution

It presents a novel recursive framework integrating dataset creation, model training, and interactive testing to enhance facial expression recognition in real-world scenarios.

Findings

The framework successfully generated the CIFE and GaMo datasets.

Fine-tuning CNN models improved recognition accuracy.

The recursive approach outperformed traditional methods on real scene data.

Abstract

In this paper, we propose a recursive framework to recognize facial expressions from images in real scenes. Unlike traditional approaches that typically focus on developing and refining algorithms for improving recognition performance on an existing dataset, we integrate three important components in a recursive manner: facial dataset generation, facial expression recognition model building, and interactive interfaces for testing and new data collection. To start with, we first create a candid-images-for-facial-expression (CIFE) dataset. We then apply a convolutional neural network (CNN) to CIFE and build a CNN model for web image expression classification. In order to increase the expression recognition accuracy, we also fine-tune the CNN model and thus obtain a better CNN facial expression recognition model. Based on the fine-tuned CNN model, we design a facial expression game engine and collect a new and more balanced dataset, GaMo. The images of this dataset are collected from the different expressions our game users make when playing the game. Finally, we evaluate the GaMo and CIFE datasets and show that our recursive framework can help build a better facial expression model for dealing with real scene facial expression tasks.