Hypernetworks That Evolve Themselves

TL;DR

This paper introduces Self-Referential Graph HyperNetworks that can evolve and adapt themselves internally, demonstrating rapid adaptation and emergent behaviors in reinforcement learning environments without external optimization.

Contribution

It presents a novel neural system that self-evolves using embedded variation mechanisms, advancing autonomous adaptation in reinforcement learning tasks.

Findings

Self-Referential GHNs adapt quickly to environmental shifts.

They evolve coherent gaits in locomotion benchmarks.

The system autonomously reduces variation around promising solutions.

Abstract

How can neural networks evolve themselves without relying on external optimizers? We propose Self-Referential Graph HyperNetworks, systems where the very machinery of variation and inheritance is embedded within the network. By uniting hypernetworks, stochastic parameter generation, and graph-based representations, Self-Referential GHNs mutate and evaluate themselves while adapting mutation rates as selectable traits. Through new reinforcement learning benchmarks with environmental shifts (CartPoleSwitch, LunarLander-Switch), Self-Referential GHNs show swift, reliable adaptation and emergent population dynamics. In the locomotion benchmark Ant-v5, they evolve coherent gaits, showing promising fine-tuning capabilities by autonomously decreasing variation in the population to concentrate around promising solutions. Our findings support the idea that evolvability itself can emerge from neural self-reference. Self-Referential GHNs reflect a step toward synthetic systems that more closely mirror biological evolution, offering tools for autonomous, open-ended learning agents.