Multi-Modal Decentralized Reinforcement Learning for Modular Reconfigurable Lunar Robots

TL;DR

This paper introduces a decentralized reinforcement learning framework enabling modular lunar robots to autonomously learn locomotion and manipulation policies, demonstrating high success rates and efficiency in simulation and field tests for reconfigurable space tasks.

Contribution

It presents a novel decentralized RL scheme allowing each module to learn its own policy, facilitating zero-shot generalization and scalable control of reconfigurable lunar robots.

Findings

Steering policy achieved 3.63° mean absolute error.

Manipulation success rate reached 84.6%.

Wheel policy reduced motor torque by 95.4%.

Abstract

Modular reconfigurable robots suit task-specific space operations, but the combinatorial growth of morphologies hinders unified control. We propose a decentralized reinforcement learning (Dec-RL) scheme where each module learns its own policy: wheel modules use Soft Actor-Critic (SAC) for locomotion and 7-DoF limbs use Proximal Policy Optimization (PPO) for steering and manipulation, enabling zero-shot generalization to unseen configurations. In simulation, the steering policy achieved a mean absolute error of 3.63{\deg} between desired and induced angles; the manipulation policy plateaued at 84.6 % success on a target-offset criterion; and the wheel policy cut average motor torque by 95.4 % relative to baseline while maintaining 99.6 % success. Lunar-analogue field tests validated zero-shot integration for autonomous locomotion, steering, and preliminary alignment for reconfiguration. The system transitioned smoothly among synchronous, parallel, and sequential modes for Policy Execution, without idle states or control conflicts, indicating a scalable, reusable, and robust approach for modular lunar robots.