Sim-to-Real Learning for Humanoid Box Loco-Manipulation

TL;DR

This paper introduces a sim-to-real reinforcement learning method enabling a humanoid robot to perform complex box manipulation tasks involving lifting, carrying, and balancing, demonstrating successful transfer from simulation to real-world execution.

Contribution

The work presents a novel sim-to-real reinforcement learning framework for humanoid box manipulation, integrating whole-body coordination and balance in a practical system.

Findings

Successful sim-to-real transfer demonstrated on humanoid robot Digit

Achieved robust box pickup and carrying across various sizes and weights

Improved balance and gait quality during manipulation tasks

Abstract

In this work we propose a learning-based approach to box loco-manipulation for a humanoid robot. This is a particularly challenging problem due to the need for whole-body coordination in order to lift boxes of varying weight, position, and orientation while maintaining balance. To address this challenge, we present a sim-to-real reinforcement learning approach for training general box pickup and carrying skills for the bipedal robot Digit. Our reward functions are designed to produce the desired interactions with the box while also valuing balance and gait quality. We combine the learned skills into a full system for box loco-manipulation to achieve the task of moving boxes from one table to another with a variety of sizes, weights, and initial configurations. In addition to quantitative simulation results, we demonstrate successful sim-to-real transfer on the humanoid r