Who Gets Credit or Blame? Attributing Accountability in Modern AI Systems

TL;DR

This paper introduces a framework for attributing accountability to different stages of AI model development, enabling identification of responsible stages for model behavior and removal of spurious correlations.

Contribution

We propose a general, efficient framework for quantifying stage-specific effects on model behavior without retraining, enhancing accountability in AI systems.

Findings

Successfully quantifies stage effects on model behavior

Identifies and removes spurious correlations in tasks

Provides a practical tool for model analysis

Abstract

Modern AI systems are typically developed through multiple stages-pretraining, fine-tuning rounds, and subsequent adaptation or alignment, where each stage builds on the previous ones and updates the model in distinct ways. This raises a critical question of accountability: when a deployed model succeeds or fails, which stage is responsible, and to what extent? We pose the accountability attribution problem for tracing model behavior back to specific stages of the model development process. To address this challenge, we propose a general framework that answers counterfactual questions about stage effects: how would the model's behavior have changed if the updates from a particular stage had not occurred? Within this framework, we introduce estimators that efficiently quantify stage effects without retraining the model, accounting for both the data and key aspects of model optimization dynamics, including learning rate schedules, momentum, and weight decay. We demonstrate that our approach successfully quantifies the accountability of each stage to the model's behavior. Based on the attribution results, our method can identify and remove spurious correlations learned during image classification and text toxicity detection tasks that were developed across multiple stages. Our approach provides a practical tool for model analysis and represents a significant step toward more accountable AI development.