Contact-Grounded Policy: Dexterous Visuotactile Policy with Generative Contact Grounding

TL;DR

This paper introduces Contact-Grounded Policy (CGP), a visuotactile policy that predicts contact states and transforms these predictions into executable commands, improving dexterous manipulation in robotics.

Contribution

The paper presents a novel contact-grounded policy that models contact states with a diffusion model and converts predictions into robot actions for better manipulation.

Findings

CGP outperforms baselines in various dexterous tasks.

It effectively predicts contact states and translates them into control commands.

The approach improves manipulation success in both real and simulated environments.

Abstract

Contact-rich dexterous manipulation with multi-finger hands remains an open challenge in robotics because task success depends on multi-point contacts that continuously evolve and are highly sensitive to object geometry, frictional transitions, and slip. Recently, tactile-informed manipulation policies have shown promise. However, most use tactile signals as additional observations rather than modeling contact state or how their action outputs interact with low-level controller dynamics. We present Contact-Grounded Policy (CGP), a visuotactile policy that grounds multi-point contacts by predicting coupled trajectories of actual robot state and tactile feedback, and using a learned contact-consistency mapping to convert these predictions into executable target robot states for a compliance controller. CGP consists of two components: (i) a conditional diffusion model that forecasts future robot state and tactile feedback in a compressed latent space, and (ii) a learned contact-consistency mapping that converts the predicted robot state-tactile pair into executable targets for a compliance controller, enabling it to realize the intended contacts. We evaluate CGP using a physical four-finger Allegro V5 hand with Digit360 fingertip tactile sensors, and a simulated five-finger Tesollo DG-5F hand with dense whole-hand tactile arrays. Across a range of dexterous tasks including in-hand manipulation, delicate grasping, and tool use, CGP outperforms visuomotor and visuotactile diffusion-policy baselines.