
TL;DR
This paper introduces a post-training stabilization method for reinforcement learning control policies by manipulating configuration paths, significantly improving their robustness against disturbances in a bipedal walker task.
Contribution
It presents a novel post-training stabilization technique based on configuration path control, enhancing RL policy robustness without retraining.
Findings
2-4 times increase in stability under perturbations
Effective stabilization applied post-training
Applicable to control policies in robotics
Abstract
Reinforcement learning methods often produce brittle policies -- policies that perform well during training, but generalize poorly beyond their direct training experience, thus becoming unstable under small disturbances. To address this issue, we propose a method for stabilizing a control policy in the space of configuration paths. It is applied post-training and relies purely on the data produced during training, as well as on an instantaneous control-matrix estimation. The approach is evaluated empirically on a planar bipedal walker subjected to a variety of perturbations. The control policies obtained via reinforcement learning are compared against their stabilized counterparts. Across different experiments, we find two- to four-fold increase in stability, when measured in terms of the perturbation amplitudes. We also provide a zero-dynamics interpretation of our approach.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsReinforcement Learning in Robotics · Model Reduction and Neural Networks · Neural Networks and Reservoir Computing
