Analysis, Interpretation, and Recognition of Facial Action Units and Expressions Using Neuro-Fuzzy Modeling

TL;DR

This paper presents a real-time neuro-fuzzy system for facial action unit recognition that is robust to variations and illumination changes, utilizing hierarchical rule-based classifiers and extensive experiments on a standard database.

Contribution

The paper introduces a novel neuro-fuzzy modeling approach for AU recognition that integrates temporal information and hierarchical classifiers for improved accuracy.

Findings

Outperforms SVM, HMM, and neural networks on Cohn-Kanade database

Robust to intensity and illumination variations

Effective in representing subtle facial expression changes

Abstract

In this paper an accurate real-time sequence-based system for representation, recognition, interpretation, and analysis of the facial action units (AUs) and expressions is presented. Our system has the following characteristics: 1) employing adaptive-network-based fuzzy inference systems (ANFIS) and temporal information, we developed a classification scheme based on neuro-fuzzy modeling of the AU intensity, which is robust to intensity variations, 2) using both geometric and appearance-based features, and applying efficient dimension reduction techniques, our system is robust to illumination changes and it can represent the subtle changes as well as temporal information involved in formation of the facial expressions, and 3) by continuous values of intensity and employing top-down hierarchical rule-based classifiers, we can develop accurate human-interpretable AU-to-expression converters. Extensive experiments on Cohn-Kanade database show the superiority of the proposed method, in comparison with support vector machines, hidden Markov models, and neural network classifiers. Keywords: biased discriminant analysis (BDA), classifier design and evaluation, facial action units (AUs), hybrid learning, neuro-fuzzy modeling.