SCE-LITE-HQ: Smooth visual counterfactual explanations with generative foundation models

TL;DR

SCE-LITE-HQ introduces a scalable, efficient framework for generating realistic and diverse counterfactual explanations in high-dimensional visual data by leveraging pretrained generative models and novel optimization techniques.

Contribution

It presents a novel method that avoids task-specific retraining, using pretrained models and smoothed gradients for high-quality counterfactuals in visual domains.

Findings

Produces valid, realistic counterfactuals

Outperforms existing methods in diversity and realism

Reduces computational overhead

Abstract

Modern neural networks achieve strong performance but remain difficult to interpret in high-dimensional visual domains. Counterfactual explanations (CFEs) provide a principled approach to interpreting black-box predictions by identifying minimal input changes that alter model outputs. However, existing CFE methods often rely on dataset-specific generative models and incur substantial computational cost, limiting their scalability to high-resolution data. We propose SCE-LITE-HQ, a scalable framework for counterfactual generation that leverages pretrained generative foundation models without task-specific retraining. The method operates in the latent space of the generator, incorporates smoothed gradients to improve optimization stability, and applies mask-based diversification to promote realistic and structurally diverse counterfactuals. We evaluate SCE-LITE-HQ on natural and medical datasets using a desiderata-driven evaluation protocol. Results show that SCE-LITE-HQ produces valid, realistic, and diverse counterfactuals competitive with or outperforming existing baselines, while avoiding the overhead of training dedicated generative models.