Diffusion-based Inverse Model of a Distributed Tactile Sensor for Object Pose Estimation

TL;DR

This paper introduces a diffusion-based inverse tactile sensor model for object pose estimation that effectively handles partial observability and multimodal contact configurations, validated in simulation and real-world scenarios.

Contribution

It presents a novel diffusion-based inverse tactile model conditioned on distributed tactile data, integrated with a particle filter for improved pose estimation without visual cues.

Findings

Enhanced sampling efficiency over baseline methods

Improved pose estimation accuracy in simulation and real-world tests

Robustness to unmodeled contact and sensor dynamics

Abstract

Tactile sensing provides a promising sensing modality for object pose estimation in manipulation settings where visual information is limited due to occlusion or environmental effects. However, efficiently leveraging tactile data for estimation remains a challenge due to partial observability, with single observations corresponding to multiple possible contact configurations. This limits conventional estimation approaches largely tailored to vision. We propose to address these challenges by learning an inverse tactile sensor model using denoising diffusion. The model is conditioned on tactile observations from a distributed tactile sensor and trained in simulation using a geometric sensor model based on signed distance fields. Contact constraints are enforced during inference through single-step projection using distance and gradient information from the signed distance field. For online pose estimation, we integrate the inverse model with a particle filter through a proposal scheme that combines generated hypotheses with particles from the prior belief. Our approach is validated in simulated and real-world planar pose estimation settings, without access to visual data or tight initial pose priors. We further evaluate robustness to unmodeled contact and sensor dynamics for pose tracking in a box-pushing scenario. Compared to local sampling baselines, the inverse sensor model improves sampling efficiency and estimation accuracy while preserving multimodal beliefs across objects with varying tactile discriminability.