DT-Loong: A Digital Twin Simulation Framework for Scalable Data Collection and Training of Humanoid Robots

TL;DR

DT-Loong is a digital twin system that enables efficient data collection and training for humanoid robots using high-fidelity simulations and real-time feedback.

Contribution

The novel contribution is the DT-Loong framework with a Priority-Guided Quadratic Optimization algorithm for real-time motion retargeting and anomaly detection.

Findings

DT-Loong generates high-quality training data using optical motion capture and motion re-targeting.

The Priority-Guided Quadratic Optimization algorithm improves mapping accuracy and reduces time delay.

The framework supports real-time environmental feedback and is suitable for monitoring and patrol applications.

Abstract

Recent advances in bionic intelligence are reshaping humanoid-robot design, demonstrating unprecedented agility, dexterity and task versatility. These breakthroughs drive an increasing need for large scale and high-quality data. Current data generation methods, however, are often expensive and time-consuming. To address this, we introduce Digital Twin Loong (DT-Loong), a digital twin system that combines a high-fidelity simulation environment with a full-scale virtual replica of the humanoid robot Loong, a bionic robot encompassing biomimetic joint design and movement mechanism. By integrating optical motion capture and human-to-humanoid motion re-targeting technologies, DT-Loong generates data for training and refining embodied AI models. We showcase the data collected from the system is of high quality. DT-Loong also proposes a Priority-Guided Quadratic Optimization algorithm for action retargeting, which achieves lower time delay and enhanced mapping accuracy. This approach enables real-time environmental feedback and anomaly detection, making it well-suited for monitoring and patrol applications. Our comprehensive framework establishes a foundation for humanoid robot training and further digital twin applications in humanoid robots to enhance their human-like behaviors through the emulation of biological systems and learning processes.

Figures