GEM: Group Enhanced Model for Learning Dynamical Control Systems

TL;DR

This paper introduces GEM, a model leveraging Lie group structures to improve learning of physical system dynamics, significantly enhancing long-term prediction and planning in control tasks.

Contribution

The paper proposes GEM, a novel approach that models dynamics on Lie algebra spaces, outperforming traditional models and improving existing systems like PETS.

Findings

GEM outperforms conventional models in long-term prediction and planning.

GEM enhances the performance of state-of-the-art systems like PETS.

Learning on Lie algebra spaces is more effective than direct state transition learning.

Abstract

Learning the dynamics of a physical system wherein an autonomous agent operates is an important task. Often these systems present apparent geometric structures. For instance, the trajectories of a robotic manipulator can be broken down into a collection of its transitional and rotational motions, fully characterized by the corresponding Lie groups and Lie algebras. In this work, we take advantage of these structures to build effective dynamical models that are amenable to sample-based learning. We hypothesize that learning the dynamics on a Lie algebra vector space is more effective than learning a direct state transition model. To verify this hypothesis, we introduce the Group Enhanced Model (GEM). GEMs significantly outperform conventional transition models on tasks of long-term prediction, planning, and model-based reinforcement learning across a diverse suite of standard continuous-control environments, including Walker, Hopper, Reacher, Half-Cheetah, Inverted Pendulums, Ant, and Humanoid. Furthermore, plugging GEM into existing state of the art systems enhances their performance, which we demonstrate on the PETS system. This work sheds light on a connection between learning of dynamics and Lie group properties, which opens doors for new research directions and practical applications along this direction. Our code is publicly available at: https://tinyurl.com/GEMMBRL.