Effective Virtual Reality Teleoperation of an Upper-body Humanoid with Modified Task Jacobians and Relaxed Barrier Functions for Self-Collision Avoidance

TL;DR

This paper introduces a novel VR teleoperation method for upper-body humanoids that uses modified task Jacobians and relaxed barrier functions to ensure self-collision avoidance, validated on real hardware with manipulation tasks.

Contribution

The paper proposes a new approach combining modified task Jacobians and relaxed barrier functions for effective self-collision-free humanoid teleoperation in VR.

Findings

Successful manipulation in tabletop tasks

Effective self-collision avoidance demonstrated

Validated on Apptronik's Astro hardware

Abstract

We present an approach for retartgeting off-the-shelf Virtual Reality (VR) trackers to effectively teleoperate an upper-body humanoid while ensuring self-collision-free motions. Key to the effectiveness was the proper assignment of trackers to joint sets via modified task Jacobians and relaxed barrier functions for self-collision avoidance. The approach was validated on Apptronik's Astro hardware by demonstrating manipulation capabilities on a table-top environment with pick-and-place box packing and a two-handed box pick up and handover task.