Tools for Verifying Neural Models' Training Data

TL;DR

This paper proposes protocols for verifying the training data of neural models, enabling verification of data provenance and training process integrity, which is crucial for trust and regulation.

Contribution

It introduces efficient verification strategies for Proof-of-Training-Data, including pre-commitment to training seeds and data inclusion detection methods.

Findings

Verification methods can detect a wide range of attacks

Protocols are compatible with current large-model training

Experimental results show effective data provenance verification

Abstract

It is important that consumers and regulators can verify the provenance of large neural models to evaluate their capabilities and risks. We introduce the concept of a "Proof-of-Training-Data": any protocol that allows a model trainer to convince a Verifier of the training data that produced a set of model weights. Such protocols could verify the amount and kind of data and compute used to train the model, including whether it was trained on specific harmful or beneficial data sources. We explore efficient verification strategies for Proof-of-Training-Data that are compatible with most current large-model training procedures. These include a method for the model-trainer to verifiably pre-commit to a random seed used in training, and a method that exploits models' tendency to temporarily overfit to training data in order to detect whether a given data-point was included in training. We show experimentally that our verification procedures can catch a wide variety of attacks, including all known attacks from the Proof-of-Learning literature.