Survival is the Only Reward: Sustainable Self-Training Through Environment-Mediated Selection

TL;DR

This paper proposes a novel self-training architecture that relies solely on environmental viability for selection, avoiding reward hacking and enabling sustainable, open-ended self-improvement in autonomous systems.

Contribution

It introduces environment-mediated selection as a new paradigm for stable self-training, demonstrating its effectiveness and unique dynamics compared to reward-based methods.

Findings

Effective strategies persist through consolidation and pruning.

Models develop meta-learning behaviors without explicit instructions.

Environment-grounded selection enables sustainable self-improvement.

Abstract

Self-training systems often degenerate due to the lack of an external criterion for judging data quality, leading to reward hacking and semantic drift. This paper provides a proof-of-concept system architecture for stable self-training under sparse external feedback and bounded memory, and empirically characterises its learning dynamics and failure modes. We introduce a self-training architecture in which learning is mediated exclusively by environmental viability, rather than by reward, objective functions, or externally defined fitness criteria. Candidate behaviours are executed under real resource constraints, and only those whose environmental effects both persist and preserve the possibility of future interaction are propagated. The environment does not provide semantic feedback, dense rewards, or task-specific supervision; selection operates solely through differential survival of behaviours as world-altering events, making proxy optimisation impossible and rendering reward-hacking evolutionarily unstable. Analysis of semantic dynamics shows that improvement arises primarily through the persistence of effective and repeatable strategies under a regime of consolidation and pruning, a paradigm we refer to as negative-space learning (NSL), and that models develop meta-learning strategies (such as deliberate experimental failure in order to elicit informative error messages) without explicit instruction. This work establishes that environment-grounded selection enables sustainable open-ended self-improvement, offering a viable path toward more robust and generalisable autonomous systems without reliance on human-curated data or complex reward shaping.