GUDA: Counterfactual Group-wise Training Data Attribution for Diffusion Models via Unlearning

TL;DR

GUDA introduces an efficient method for group-wise data attribution in diffusion models by approximating counterfactuals through unlearning, enabling reliable influence estimation without costly retraining.

Contribution

It proposes GUDA, a novel unlearning-based approach that approximates counterfactual models for group attribution, significantly reducing computational costs compared to LOGO retraining.

Findings

GUDA outperforms semantic similarity and gradient-based methods in identifying influential groups.

It achieves 100x speedup over LOGO retraining on CIFAR-10.

GUDA reliably attributes artistic styles in diffusion models.

Abstract

Training-data attribution for vision generative models aims to identify which training data influenced a given output. While most methods score individual examples, practitioners often need group-level answers (e.g., artistic styles or object classes). Group-wise attribution is counterfactual: how would a model's behavior on a generated sample change if a group were absent from training? A natural realization of this counterfactual is Leave-One-Group-Out (LOGO) retraining, which retrains the model with each group removed; however, it becomes computationally prohibitive as the number of groups grows. We propose GUDA (Group Unlearning-based Data Attribution) for diffusion models, which approximates each counterfactual model by applying machine unlearning to a shared full-data model instead of training from scratch. GUDA quantifies group influence using differences in a likelihood-based scoring rule (ELBO) between the full model and each unlearned counterfactual. Experiments on CIFAR-10 and artistic style attribution with Stable Diffusion show that GUDA identifies primary contributing groups more reliably than semantic similarity, gradient-based attribution, and instance-level unlearning approaches, while achieving x100 speedup on CIFAR-10 over LOGO retraining.