How Many Visual Levers Drive Urban Perception? Interventional Counterfactuals via Multiple Localised Edits

TL;DR

This paper introduces a framework for understanding how localized visual changes in street-view images influence human perception of safety, using structured counterfactual edits and explainability techniques.

Contribution

It presents a novel lever-based interventional counterfactual approach that generates and evaluates semantic visual edits to interpret perception models.

Findings

Identified patterns in how specific visual features affect safety perception.

Demonstrated the framework on 50 scenes across five cities.

Revealed limitations and potential directions for future validation.

Abstract

Street-view perception models predict subjective attributes such as safety at scale, but remain correlational: they do not identify which localized visual changes would plausibly shift human judgement for a specific scene. We propose a lever-based interventional counterfactual framework that recasts scene-level explainability as a bounded search over structured counterfactual edits. Each lever specifies a semantic concept, spatial support, intervention direction, and constrained edit template. Candidate edits are generated through prompt-conditioned image editing and retained only if they satisfy validity checks for same-place preservation, locality, realism, and plausibility. In a pilot across 50 scenes from five cities, the framework reveals preliminary proxy-based directional patterns and a practical failure taxonomy under prompt-only editing, with Mobility Infrastructure and Physical Maintenance showing the largest auxiliary safety shifts. Human pairwise judgements remain the ground-truth endpoint for future validation.