Optimal Gait Control for a Tendon-driven Soft Quadruped Robot by Model-based Reinforcement Learning

TL;DR

This paper introduces a model-based reinforcement learning approach to optimize gait control for a soft quadruped robot with tendon-driven actuators, achieving improved efficiency, robustness, and adaptability over benchmark methods.

Contribution

It develops a multi-stage MBRL framework tailored for soft quadruped robots, enhancing gait performance and robustness compared to previous model-free approaches.

Findings

MBRL significantly improves gait control efficiency.

The developed policy is robust and adaptable to deformable morphology.

Performance surpasses benchmark methods in stability and speed.

Abstract

This study presents an innovative approach to optimal gait control for a soft quadruped robot enabled by four Compressible Tendon-driven Soft Actuators (CTSAs). Improving our previous studies of using model-free reinforcement learning for gait control, we employ model-based reinforcement learning (MBRL) to further enhance the performance of the gait controller. Compared to rigid robots, the proposed soft quadruped robot has better safety, less weight, and a simpler mechanism for fabrication and control. However, the primary challenge lies in developing sophisticated control algorithms to attain optimal gait control for fast and stable locomotion. The research employs a multi-stage methodology, including state space restriction, data-driven model training, and reinforcement learning algorithm development. Compared to benchmark methods, the proposed MBRL algorithm, combined with post-training, significantly improves the efficiency and performance of gait control policies. The developed policy is both robust and adaptable to the robot's deformable morphology. The study concludes by highlighting the practical applicability of these findings in real-world scenarios.