Credit Attribution and Stable Compression

TL;DR

This paper introduces new stability-based definitions for credit attribution in machine learning, allowing certain data points to influence models non-stably with permission, extending differential privacy concepts.

Contribution

It proposes relaxed stability notions for credit attribution, extending differential privacy, and characterizes their learnability within the PAC framework.

Findings

Extended stability notions encompass differential privacy and sample compression.

Characterized learnability of algorithms under these new stability definitions.

Provided future research directions in credit attribution and stability.

Abstract

Credit attribution is crucial across various fields. In academic research, proper citation acknowledges prior work and establishes original contributions. Similarly, in generative models, such as those trained on existing artworks or music, it is important to ensure that any generated content influenced by these works appropriately credits the original creators. We study credit attribution by machine learning algorithms. We propose new definitions--relaxations of Differential Privacy--that weaken the stability guarantees for a designated subset of $k$ datapoints. These $k$ datapoints can be used non-stably with permission from their owners, potentially in exchange for compensation. Meanwhile, the remaining datapoints are guaranteed to have no significant influence on the algorithm's output. Our framework extends well-studied notions of stability, including Differential Privacy ($k = 0$), differentially private learning with public data (where the $k$ public datapoints are fixed in advance), and stable sample compression (where the $k$ datapoints are selected adaptively by the algorithm). We examine the expressive power of these stability notions within the PAC learning framework, provide a comprehensive characterization of learnability for algorithms adhering to these principles, and propose directions and questions for future research.