MIRROR: Visual Motion Imitation via Real-time Retargeting and Teleoperation with Parallel Differential Inverse Kinematics

TL;DR

This paper introduces MIRROR, a real-time humanoid teleoperation system using a GPU-accelerated, continuation-based differential inverse kinematics method that enhances safety, stability, and responsiveness during task retargeting and collision avoidance.

Contribution

It presents a novel GPU-parallelized continuation-based differential IK approach that improves escape from local minima and integrates collision avoidance for real-time humanoid teleoperation.

Findings

Achieves real-time upper-body teleoperation on THEMS humanoid robot.

Effectively avoids joint limits and collisions during tasks.

Maintains responsiveness and safety in complex environments.

Abstract

Real-time humanoid teleoperation requires inverse kinematics (IK) solvers that are both responsive and constraint-safe under kinematic redundancy and self-collision constraints. While differential IK enables efficient online retargeting, its locally linearized updates are inherently basin-dependent and often become trapped near joint limits, singularities, or active collision boundaries, leading to unsafe or stagnant behavior. We propose a GPU-parallelized, continuation-based differential IK that improves escape from such constraint-induced local minima while preserving real-time performance, promoting safety and stability. Multiple constrained IK quadratic programs are evaluated in parallel, together with a self-collision avoidance control barrier function (CBF), and a Lyapunov-based progression criterion selects updates that reduce the final global task-space error. The method is paired with a visual skeletal pose estimation pipeline that enables robust, real-time upper-body teleoperation on the THEMIS humanoid robot hardware in real-world tasks.