PDFed: Privacy-Preserving and Decentralized Asynchronous Federated Learning for Diffusion Models

TL;DR

PDFed introduces a decentralized, privacy-preserving federated learning protocol for diffusion models that reduces data memorization, supports asynchronous collaboration, and enhances transparency using blockchain technology.

Contribution

It presents a novel blockchain-based federated learning protocol for diffusion models that minimizes data memorization and enables asynchronous, decentralized training.

Findings

Reduces private data memorization in diffusion models

Supports asynchronous collaboration among participants

Enhances transparency and auditability of AI model provenance

Abstract

We present PDFed, a decentralized, aggregator-free, and asynchronous federated learning protocol for training image diffusion models using a public blockchain. In general, diffusion models are prone to memorization of training data, raising privacy and ethical concerns (e.g., regurgitation of private training data in generated images). Federated learning (FL) offers a partial solution via collaborative model training across distributed nodes that safeguard local data privacy. PDFed proposes a novel sample-based score that measures the novelty and quality of generated samples, incorporating these into a blockchain-based federated learning protocol that we show reduces private data memorization in the collaboratively trained model. In addition, PDFed enables asynchronous collaboration among participants with varying hardware capabilities, facilitating broader participation. The protocol records the provenance of AI models, improving transparency and auditability, while also considering automated incentive and reward mechanisms for participants. PDFed aims to empower artists and creators by protecting the privacy of creative works and enabling decentralized, peer-to-peer collaboration. The protocol positively impacts the creative economy by opening up novel revenue streams and fostering innovative ways for artists to benefit from their contributions to the AI space.