SixthSense: Task-Agnostic Proprioception-Only Whole-Body Wrench Estimation for Humanoids

TL;DR

SixthSense is a novel proprioception-only method for humanoid robots that accurately estimates contact events and wrenches without relying on external sensors, enhancing force-interaction capabilities.

Contribution

It introduces a task-agnostic, proprioception-based approach using conditional flow matching to infer contact information, overcoming limitations of existing analytical methods.

Findings

Achieved high accuracy in contact detection across various humanoid behaviors.

Enabled reliable force estimation solely from proprioception and IMU data.

Demonstrated effectiveness in collision detection and human-robot interaction scenarios.

Abstract

Humanoid robots are entering our physical world at scale, yet as oversized toys--good at singing and dancing, but short on force-interaction capabilities for practical tasks. Bridging this gap necessitates prioritizing reliable contact perception as a fundamental requirement. Estimating external wrenches in humanoids is complicated by floating-base dynamics and indeterminate contact locations. Existing analytical frameworks require idealistic assumptions and hard-to-obtain measurements, which are often unavailable in practice. To bridge this gap, we propose SixthSense, a task-agnostic approach that infers whole-body contact timing, location, and wrenches from proprioception and IMU data alone. To capture the multi-modal dynamics between unstructured contact inputs and the uncertain motion outputs, we employ conditional flow matching to tokenize proprioceptive histories and estimate a spatiotemporally sparse contact-event flow. SixthSense serves as a plug-and-play perception module for applications including collision detection, physical human-robot interaction, and force-feedback teleoperation. Experiments across standing, walking, and whole-body motion-tracking policies showcased unprecedented performance in diverse behaviors.