THIN: THrowable Information Networks and Application for Facial Expression Recognition In The Wild

TL;DR

THIN introduces a novel framework that leverages exogenous variables to improve facial expression recognition in the wild by creating invariant representations, outperforming existing methods across multiple datasets.

Contribution

The paper proposes THIN, a method that uses dual exogenous/endogenous representations and an exogenous dispelling loss to enhance invariance in facial expression recognition.

Findings

THIN outperforms state-of-the-art methods on several FER datasets.

The approach effectively removes identity bias in facial expression recognition.

THIN generalizes well to other recognition tasks with exogenous variables.

Abstract

For a number of machine learning problems, an exogenous variable can be identified such that it heavily influences the appearance of the different classes, and an ideal classifier should be invariant to this variable. An example of such exogenous variable is identity if facial expression recognition (FER) is considered. In this paper, we propose a dual exogenous/endogenous representation. The former captures the exogenous variable whereas the second one models the task at hand (e.g. facial expression). We design a prediction layer that uses a tree-gated deep ensemble conditioned by the exogenous representation. We also propose an exogenous dispelling loss to remove the exogenous information from the endogenous representation. Thus, the exogenous information is used two times in a throwable fashion, first as a conditioning variable for the target task, and second to create invariance within the endogenous representation. We call this method THIN, standing for THrowable Information Networks. We experimentally validate THIN in several contexts where an exogenous information can be identified, such as digit recognition under large rotations and shape recognition at multiple scales. We also apply it to FER with identity as the exogenous variable. We demonstrate that THIN significantly outperforms state-of-the-art approaches on several challenging datasets.