MAGNET: Autonomous Expert Model Generation via Decentralized Autoresearch and BitNet Training

TL;DR

MAGNET is a decentralized system that autonomously generates, trains, and deploys domain-specific language models using autonomous research, bit-level training, distributed merging, and on-chain tracking, validated across multiple case studies.

Contribution

It introduces MAGNET, a novel autonomous framework combining autoresearch, BitNet training, distributed merging, and blockchain tracking for domain-specific language models.

Findings

Achieved high accuracy in video safety classification (up to 0.9851).

Improved cryptocurrency prediction hit rate from 41% to 54.9%.

Optimized BitNet hyperparameters, reducing validation loss by 16.7%.

Abstract

We present MAGNET (Model Autonomously Growing Network), a decentralized system for autonomous generation, training, and serving of domain-expert language models across commodity hardware. MAGNET integrates four components: (1) autoresearch, an autonomous ML research pipeline that automates dataset generation, hyperparameter exploration, evaluation, and error-driven iteration; (2) BitNet b1.58 ternary training, enabling CPU-native inference via bitnet.cpp without GPU hardware; (3) DiLoCo-based distributed merging for communication-efficient aggregation of domain specialists; and (4) on-chain contribution tracking on the HOOTi EVM chain. We validate autoresearch through three case studies: video safety classification (balanced accuracy 0.9287 to 0.9851), cryptocurrency directional prediction (41% to 54.9% hit rate), and BitNet hyperparameter optimization (10-phase sweep, -16.7% validation loss).