Incentivizing Permissionless Distributed Learning of LLMs

TL;DR

This paper presents Gauntlet, an incentive system deployed on the bittensor blockchain that enables permissionless distributed training of large language models, rewarding peer contributions and ensuring fair participation.

Contribution

The paper introduces Gauntlet, a novel incentive mechanism for permissionless distributed deep learning of LLMs, including a live deployment training a 1.2B model.

Findings

Successfully trained a 1.2B parameter LLM using permissionless pseudo-gradient contributions.

Implemented a real-world incentive system that rewards participants with tokens based on contribution value.

Demonstrated the effectiveness of the filtering and evaluation mechanisms in a live blockchain environment.

Abstract

We describe an incentive system for distributed deep learning of foundational models where peers are rewarded for contributions. The incentive system, \textit{Gauntlet}, has been deployed on the bittensor blockchain and used to train a 1.2B LLM with completely permissionless contributions of pseudo-gradients: no control over the users that can register or their hardware. \textit{Gauntlet} can be applied to any synchronous distributed training scheme that relies on aggregating updates or pseudo-gradients. We rely on a two-stage mechanism for fast filtering of peer uptime, reliability, and synchronization, combined with the core component that estimates the loss before and after individual pseudo-gradient contributions. We utilized an OpenSkill rating system to track competitiveness of pseudo-gradient scores across time. Finally, we introduce a novel mechanism to ensure peers on the network perform unique computations. Our live 1.2B run, which has paid out real-valued tokens to participants based on the value of their contributions, yielded a competitive (on a per-iteration basis) 1.2B model that demonstrates the utility of our incentive system.