QuadFM: Foundational Text-Driven Quadruped Motion Dataset for Generation and Control

TL;DR

QuadFM is a comprehensive, high-fidelity dataset of quadruped motions with rich language annotations, enabling advanced text-to-motion generation and control for more intuitive human-robot interactions.

Contribution

The paper introduces QuadFM, the first large-scale dataset combining diverse quadruped motions with detailed language grounding, and proposes Gen2Control RL for real-time, end-to-end motion synthesis on edge hardware.

Findings

Achieves real-time motion synthesis with <500 ms latency.

Supports diverse, realistic quadruped behaviors including emotional and interactive motions.

Demonstrates effective transfer from simulation to real-world robot control.

Abstract

Despite significant advances in quadrupedal robotics, a critical gap persists in foundational motion resources that holistically integrate diverse locomotion, emotionally expressive behaviors, and rich language semantics-essential for agile, intuitive human-robot interaction. Current quadruped motion datasets are limited to a few mocap primitives (e.g., walk, trot, sit) and lack diverse behaviors with rich language grounding. To bridge this gap, we introduce Quadruped Foundational Motion (QuadFM) , the first large-scale, ultra-high-fidelity dataset designed for text-to-motion generation and general motion control. QuadFM contains 11,784 curated motion clips spanning locomotion, interactive, and emotion-expressive behaviors (e.g., dancing, stretching, peeing), each with three-layer annotation-fine-grained action labels, interaction scenarios, and natural language commands-totaling 35,352 descriptions to support language-conditioned understanding and command execution. We further propose Gen2Control RL, a unified framework that jointly trains a general motion controller and a text-to-motion generator, enabling efficient end-to-end inference on edge hardware. On a real quadruped robot with an NVIDIA Orin, our system achieves real-time motion synthesis (<500 ms latency). Simulation and real-world results show realistic, diverse motions while maintaining robust physical interaction. The dataset will be released at https://github.com/GaoLii/QuadFM.