Towards Unbiased Visual Emotion Recognition via Causal Intervention

TL;DR

This paper introduces a causal inference-based method called IERN to reduce dataset bias in visual emotion recognition, improving robustness and generalization across benchmarks.

Contribution

The paper proposes a novel IERN model that performs backdoor adjustment by disentangling confounders, addressing dataset bias in emotion recognition tasks.

Findings

IERN outperforms state-of-the-art methods on three emotion benchmarks.

The approach effectively reduces the influence of dataset bias.

Experimental results validate the causal intervention strategy.

Abstract

Although much progress has been made in visual emotion recognition, researchers have realized that modern deep networks tend to exploit dataset characteristics to learn spurious statistical associations between the input and the target. Such dataset characteristics are usually treated as dataset bias, which damages the robustness and generalization performance of these recognition systems. In this work, we scrutinize this problem from the perspective of causal inference, where such dataset characteristic is termed as a confounder which misleads the system to learn the spurious correlation. To alleviate the negative effects brought by the dataset bias, we propose a novel Interventional Emotion Recognition Network (IERN) to achieve the backdoor adjustment, which is one fundamental deconfounding technique in causal inference. Specifically, IERN starts by disentangling the dataset-related context feature from the actual emotion feature, where the former forms the confounder. The emotion feature will then be forced to see each confounder stratum equally before being fed into the classifier. A series of designed tests validate the efficacy of IERN, and experiments on three emotion benchmarks demonstrate that IERN outperforms state-of-the-art approaches for unbiased visual emotion recognition. Code is available at https://github.com/donydchen/causal_emotion