HAFO: A Force-Adaptive Control Framework for Humanoid Robots in Intense Interaction Environments

TL;DR

This paper introduces HAFO, a dual-agent reinforcement learning framework enabling humanoid robots to perform robust and precise force-interaction tasks through coupled training and disturbance modeling.

Contribution

The paper presents a novel dual-agent RL framework with a constrained action space and disturbance modeling for improved force-interaction control in humanoid robots.

Findings

Achieves stable whole-body control under diverse force conditions

Demonstrates robustness in load-bearing and disturbance scenarios

Maintains stability even in rope suspension environments

Abstract

Reinforcement learning (RL) controllers have made impressive progress in humanoid locomotion and light-weight object manipulation. However, achieving robust and precise motion control with intense force interaction remains a significant challenge. To address these limitations, this paper proposes HAFO, a dual-agent reinforcement learning framework that concurrently optimizes both a robust locomotion strategy and a precise upper-body manipulation strategy via coupled training. We employ a constrained residual action space to improve dual-agent training stability and sample efficiency. The external tension disturbances are explicitly modeled using a spring-damper system, allowing for fine-grained force control through manipulation of the virtual spring. In this process, the reinforcement learning policy autonomously generates a disturbance-rejection response by utilizing environmental feedback. The experimental results demonstrate that HAFO achieves whole-body control for humanoid robots across diverse force-interaction environments using a single dual-agent policy, delivering outstanding performance under load-bearing and thrust-disturbance conditions, while maintaining stable operation even under rope suspension state.