Neural Internal Model Control: Learning a Robust Control Policy via Predictive Error Feedback

TL;DR

This paper introduces Neural Internal Model Control, a hybrid approach combining model-based and reinforcement learning techniques to improve robustness and generalization in robotic motion control, demonstrated on quadrotors and quadrupeds.

Contribution

The paper presents a novel framework that integrates predictive error feedback with RL and model-based control, simplifying the internal model for enhanced robustness.

Findings

Outperforms state-of-the-art methods on robotic control tasks

Demonstrates robustness in real-world quadrotor payload experiments

Achieves effective sim-to-real transfer in complex environments

Abstract

Accurate motion control in the face of disturbances within complex environments remains a major challenge in robotics. Classical model-based approaches often struggle with nonlinearities and unstructured disturbances, while RL-based methods can be fragile when encountering unseen scenarios. In this paper, we propose a novel framework, Neural Internal Model Control, which integrates model-based control with RL-based control to enhance robustness. Our framework streamlines the predictive model by applying Newton-Euler equations for rigid-body dynamics, eliminating the need to capture complex high-dimensional nonlinearities. This internal model combines model-free RL algorithms with predictive error feedback. Such a design enables a closed-loop control structure to enhance the robustness and generalizability of the control system. We demonstrate the effectiveness of our framework on both quadrotors and quadrupedal robots, achieving superior performance compared to state-of-the-art methods. Furthermore, real-world deployment on a quadrotor with rope-suspended payloads highlights the framework's robustness in sim-to-real transfer. Our code is released at https://github.com/thu-uav/NeuralIMC.