Reinforcement Learning Enabled Adaptive Multi-Task Control for Bipedal Soccer Robots

TL;DR

This paper presents a modular reinforcement learning framework for adaptive multi-task control in bipedal soccer robots, enabling stable gait generation, task switching, and rapid fall recovery in dynamic environments.

Contribution

It introduces a novel RL-based control architecture combining gait generation with task-specific networks and a posture-driven state machine for seamless multi-task operation.

Findings

Successfully demonstrated in Unity simulations with reliable ball kicking.

Achieved rapid fall recovery with an average time of 0.715 seconds.

Enhanced spatial adaptability in complex scenarios.

Abstract

Developing bipedal football robots in dynamiccombat environments presents challenges related to motionstability and deep coupling of multiple tasks, as well ascontrol switching issues between different states such as up-right walking and fall recovery. To address these problems,this paper proposes a modular reinforcement learning (RL)framework for achieving adaptive multi-task control. Firstly,this framework combines an open-loop feedforward oscilla-tor with a reinforcement learning-based feedback residualstrategy, effectively separating the generation of basic gaitsfrom complex football actions. Secondly, a posture-driven statemachine is introduced, clearly switching between the ballseeking and kicking network (BSKN) and the fall recoverynetwork (FRN), fundamentally preventing state interference.The FRN is efficiently trained through a progressive forceattenuation curriculum learning strategy. The architecture wasverified in Unity simulations of bipedal robots, demonstratingexcellent spatial adaptability-reliably finding and kicking theball even in restricted corner scenarios-and rapid autonomousfall recovery (with an average recovery time of 0.715 seconds).This ensures seamless and stable operation in complex multi-task environments.