Gray-Box Computed Torque Control for Differential-Drive Mobile Robot Tracking

TL;DR

This paper introduces a gray-box control method combining computed torque control with deep reinforcement learning to improve tracking performance and stability of differential-drive robots with fewer learning episodes.

Contribution

The paper proposes a novel hybrid control approach that integrates computed torque control with deep reinforcement learning, enhancing sample efficiency and stability in robot tracking tasks.

Findings

The method achieves accurate tracking with fewer learning episodes.

It ensures closed-loop stability through known parameter constraints.

Performance surpasses traditional controllers in simulation.

Abstract

This study presents a learning-based nonlinear algorithm for tracking control of differential-drive mobile robots. The Computed Torque Method (CTM) suffers from inaccurate knowledge of system parameters, while Deep Reinforcement Learning (DRL) algorithms are known for sample inefficiency and weak stability guarantees. The proposed method replaces the black-box policy network of a DRL agent with a gray-box Computed Torque Controller (CTC) to improve sample efficiency and ensure closed-loop stability. This approach enables finding an optimal set of controller parameters for an arbitrary reward function using only a few short learning episodes. The Twin-Delayed Deep Deterministic Policy Gradient (TD3) algorithm is used for this purpose. Additionally, some controller parameters are constrained to lie within known value ranges, ensuring the RL agent learns physically plausible values. A technique is also applied to enforce a critically damped closed-loop time response. The controller's performance is evaluated on a differential-drive mobile robot simulated in the MuJoCo physics engine and compared against the raw CTC and a conventional kinematic controller.