Counterfactual Attention Learning for Fine-Grained Visual Categorization and Re-identification

TL;DR

This paper introduces a counterfactual attention learning approach for fine-grained visual recognition, leveraging causal inference to improve attention quality and enhance recognition accuracy across various tasks.

Contribution

It proposes a novel counterfactual causality-based attention learning method that outperforms traditional likelihood-based approaches in fine-grained recognition tasks.

Findings

Consistent improvement across multiple benchmarks

Effective attention quality measurement via counterfactual analysis

Enhanced performance in fine-grained categorization and re-identification

Abstract

Attention mechanism has demonstrated great potential in fine-grained visual recognition tasks. In this paper, we present a counterfactual attention learning method to learn more effective attention based on causal inference. Unlike most existing methods that learn visual attention based on conventional likelihood, we propose to learn the attention with counterfactual causality, which provides a tool to measure the attention quality and a powerful supervisory signal to guide the learning process. Specifically, we analyze the effect of the learned visual attention on network prediction through counterfactual intervention and maximize the effect to encourage the network to learn more useful attention for fine-grained image recognition. Empirically, we evaluate our method on a wide range of fine-grained recognition tasks where attention plays a crucial role, including fine-grained image categorization, person re-identification, and vehicle re-identification. The consistent improvement on all benchmarks demonstrates the effectiveness of our method. Code is available at https://github.com/raoyongming/CAL