Discovering Causal Signals in Images

TL;DR

This paper introduces a method to identify causal signals in images by classifying causal directions between object features and context, revealing observable footprints of causal dispositions in static images.

Contribution

It presents a novel approach combining causal discovery with image analysis to distinguish object features from contextual features based on causal direction classification.

Findings

Existence of observable causal footprints in images.

Causal direction correlates with object versus context features.

State-of-the-art accuracy in causal direction classification.

Abstract

This paper establishes the existence of observable footprints that reveal the "causal dispositions" of the object categories appearing in collections of images. We achieve this goal in two steps. First, we take a learning approach to observational causal discovery, and build a classifier that achieves state-of-the-art performance on finding the causal direction between pairs of random variables, given samples from their joint distribution. Second, we use our causal direction classifier to effectively distinguish between features of objects and features of their contexts in collections of static images. Our experiments demonstrate the existence of a relation between the direction of causality and the difference between objects and their contexts, and by the same token, the existence of observable signals that reveal the causal dispositions of objects.