Recursive Counterfactual Deconfounding for Object Recognition

TL;DR

This paper introduces RCD, a recursive counterfactual deconfounding model for object recognition that improves discriminability and generalization by iteratively learning and removing confounders using counterfactual analysis.

Contribution

The paper presents a novel recursive counterfactual deconfounding framework that enhances object recognition performance by explicitly modeling and eliminating confounders.

Findings

Outperforms 11 state-of-the-art methods in recognition tasks

Effective in both closed-set and open-set scenarios

Improves discriminability and generalization through recursive learning

Abstract

Image recognition is a classic and common task in the computer vision field, which has been widely applied in the past decade. Most existing methods in literature aim to learn discriminative features from labeled images for classification, however, they generally neglect confounders that infiltrate into the learned features, resulting in low performances for discriminating test images. To address this problem, we propose a Recursive Counterfactual Deconfounding model for object recognition in both closed-set and open-set scenarios based on counterfactual analysis, called RCD. The proposed model consists of a factual graph and a counterfactual graph, where the relationships among image features, model predictions, and confounders are built and updated recursively for learning more discriminative features. It performs in a recursive manner so that subtler counterfactual features could be learned and eliminated progressively, and both the discriminability and generalization of the proposed model could be improved accordingly. In addition, a negative correlation constraint is designed for alleviating the negative effects of the counterfactual features further at the model training stage. Extensive experimental results on both closed-set recognition task and open-set recognition task demonstrate that the proposed RCD model performs better than 11 state-of-the-art baselines significantly in most cases.