Whole-Body Coordination for Dynamic Object Grasping with Legged Manipulators

TL;DR

This paper introduces a new benchmark and a novel neural network framework enabling quadrupedal robots with manipulators to effectively grasp dynamic objects in unstructured environments, advancing the field of dynamic whole-body coordination.

Contribution

The paper presents DQ-Bench for evaluating dynamic grasping and DQ-Net, a teacher-student framework that infers grasp configurations using limited perceptual cues, improving dynamic object grasping capabilities.

Findings

DQ-Net outperforms baseline methods in success rate.

The framework demonstrates robustness across various task settings.

Extensive experiments validate the effectiveness of the proposed approach.

Abstract

Quadrupedal robots with manipulators offer strong mobility and adaptability for grasping in unstructured, dynamic environments through coordinated whole-body control. However, existing research has predominantly focused on static-object grasping, neglecting the challenges posed by dynamic targets and thus limiting applicability in dynamic scenarios such as logistics sorting and human-robot collaboration. To address this, we introduce DQ-Bench, a new benchmark that systematically evaluates dynamic grasping across varying object motions, velocities, heights, object types, and terrain complexities, along with comprehensive evaluation metrics. Building upon this benchmark, we propose DQ-Net, a compact teacher-student framework designed to infer grasp configurations from limited perceptual cues. During training, the teacher network leverages privileged information to holistically model both the static geometric properties and dynamic motion characteristics of the target, and integrates a grasp fusion module to deliver robust guidance for motion planning. Concurrently, we design a lightweight student network that performs dual-viewpoint temporal modeling using only the target mask, depth map, and proprioceptive state, enabling closed-loop action outputs without reliance on privileged data. Extensive experiments on DQ-Bench demonstrate that DQ-Net achieves robust dynamic objects grasping across multiple task settings, substantially outperforming baseline methods in both success rate and responsiveness.