Counterfactual Explanations on Robust Perceptual Geodesics

TL;DR

This paper introduces Perceptual Counterfactual Geodesics (PCG), a method that generates semantically meaningful counterfactual explanations by tracing geodesics in a perceptually aligned Riemannian space, improving on existing approaches.

Contribution

The paper proposes a novel geodesic-based approach using a perceptual Riemannian metric for counterfactual explanations, addressing limitations of previous flat or misaligned geometry methods.

Findings

PCG outperforms baseline methods on three vision datasets.

It produces smooth, on-manifold, semantically valid transitions.

Reveals failure modes hidden under standard metrics.

Abstract

Latent-space optimization methods for counterfactual explanations - framed as minimal semantic perturbations that change model predictions - inherit the ambiguity of Wachter et al.'s objective: the choice of distance metric dictates whether perturbations are meaningful or adversarial. Existing approaches adopt flat or misaligned geometries, leading to off-manifold artifacts, semantic drift, or adversarial collapse. We introduce Perceptual Counterfactual Geodesics (PCG), a method that constructs counterfactuals by tracing geodesics under a perceptually Riemannian metric induced from robust vision features. This geometry aligns with human perception and penalizes brittle directions, enabling smooth, on-manifold, semantically valid transitions. Experiments on three vision datasets show that PCG outperforms baselines and reveals failure modes hidden under standard metrics.