Hereditary Geometric Meta-RL: Nonlocal Generalization via Task Symmetries

TL;DR

This paper introduces a geometric approach to Meta-RL that leverages task symmetries for broader generalization, moving beyond local smoothness to discover and utilize inherent system symmetries for improved learning and inference.

Contribution

The paper develops a hereditary geometric framework for Meta-RL that exploits system symmetries, along with a differential symmetry discovery method for efficient structure learning.

Findings

Successfully recovers ground-truth symmetries in experiments

Achieves wider task space generalization than baseline methods

Improves numerical stability and sample efficiency

Abstract

Meta-Reinforcement Learning (Meta-RL) commonly generalizes via smoothness in the task encoding. While this enables local generalization around each training task, it requires dense coverage of the task space and leaves richer task space structure untapped. In response, we develop a geometric perspective that endows the task space with a "hereditary geometry" induced by the inherent symmetries of the underlying system. Concretely, the agent reuses a policy learned at the train time by transforming states and actions through actions of a Lie group. This converts Meta-RL into symmetry discovery rather than smooth extrapolation, enabling the agent to generalize to wider regions of the task space. We show that when the task space is inherited from the symmetries of the underlying system, the task space embeds into a subgroup of those symmetries whose actions are linearizable, connected, and compact--properties that enable efficient learning and inference at the test time. To learn these structures, we develop a differential symmetry discovery method. This collapses functional invariance constraints and thereby improves numerical stability and sample efficiency over functional approaches. Empirically, on a two-dimensional navigation task, our method efficiently recovers the ground-truth symmetry and generalizes across the entire task space, while a common baseline generalizes only near training tasks.