DARWIN: Dynamic Agentically Rewriting Self-Improving Network

TL;DR

DARWIN is an evolutionary GPT training framework where multiple agents iteratively modify training code, leading to performance improvements demonstrated by increased efficiency and reduced perplexity over several iterations.

Contribution

The paper introduces a novel self-improving GPT training method using genetic algorithms and multi-agent code modification, with integrated HITL and memory tracking.

Findings

Achieved 1.26% improvement in FLOPS utilization

Achieved 2.07% reduction in perplexity

Demonstrated scalability over multiple iterations

Abstract

DARWIN is an evolutionary GPT model, utilizing a genetic-algorithm like optimization structure with several independent GPT agents being trained individually using unique training code. Each iteration, the GPT models are prompted to modify the training code of one another in an attempt to improve their performance in a mutation-like manner, and the best GPT agents are then benchmarked and selected for the next iteration by genetic algorithm. For demonstration purposes and due to budget and time constraints, OpenAI API is used to prompt training code improvements and the nanoGPT framework is used as the training code. DARWIN also utilizes persistent JSON-based memory files to track previous reasoning and changes to code to correlate with improvement to model performance. and a bidirectional interface for HITL intervention allowing the model to request upgrades such as additional datasets, training scripts, and restructuring of file hierarchies. In experiments, DARWIN achieved a 1.26 percent improvement in model FLOPS utilization (MFU) and a 2.07 percent improvement to perplexity in 5 iterations of training over baseline configurations, demonstrating promising capabilities as a foundation for scaling evolutionary GPT training.