Counterfactual Generative Networks

TL;DR

This paper introduces a generative model that disentangles causal factors in images, enabling the creation of counterfactual images to improve robustness and interpretability in classifiers.

Contribution

It proposes a causal generative framework that decomposes images into independent mechanisms, allowing for counterfactual image generation without supervision.

Findings

Counterfactual images improve out-of-distribution robustness.

Model can generate images on MNIST and ImageNet.

Efficient training on a single GPU using pre-trained models.

Abstract

Neural networks are prone to learning shortcuts -- they often model simple correlations, ignoring more complex ones that potentially generalize better. Prior works on image classification show that instead of learning a connection to object shape, deep classifiers tend to exploit spurious correlations with low-level texture or the background for solving the classification task. In this work, we take a step towards more robust and interpretable classifiers that explicitly expose the task's causal structure. Building on current advances in deep generative modeling, we propose to decompose the image generation process into independent causal mechanisms that we train without direct supervision. By exploiting appropriate inductive biases, these mechanisms disentangle object shape, object texture, and background; hence, they allow for generating counterfactual images. We demonstrate the ability of our model to generate such images on MNIST and ImageNet. Further, we show that the counterfactual images can improve out-of-distribution robustness with a marginal drop in performance on the original classification task, despite being synthetic. Lastly, our generative model can be trained efficiently on a single GPU, exploiting common pre-trained models as inductive biases.