Adaptive Manipulation Potential and Haptic Estimation for Tool-Mediated Interaction

TL;DR

This paper presents a unified framework combining haptic estimation, online planning, and adaptive stiffness control for tool-mediated manipulation, enabling robust contact-rich interactions despite visual occlusion.

Contribution

It introduces a parameterized Equilibrium Manifold and a hybrid inference strategy for haptic SLAM, advancing adaptive manipulation and real-time physical interaction modeling.

Findings

Successful validation in simulation and 260+ real-world screw-loosening trials.

Haptic SLAM improves shape and pose estimation accuracy.

Adaptive stiffness control prevents jamming in tight interactions.

Abstract

Achieving human-level dexterity in contact-rich, tool-mediated manipulation remains a significant challenge due to visual occlusion and the underdetermined nature of haptic sensing. This paper introduces a parameterized Equilibrium Manifold (EM) as a unified representation for tool-mediated interaction, and develops a closed-loop framework that integrates haptic estimation, online planning, and adaptive stiffness control. We establish a physical-geometric duality using an adaptive manipulation potential incorporating a differentiable contact model, which induces the manifold's geometric structure and ensures that complex physical interactions are encapsulated as continuous operations on the EM. Within this framework, we reformulate haptic estimation as a manifold parameter estimation problem. Specifically, a hybrid inference strategy (haptic SLAM) is employed in which discrete object shapes are classified via particle filtering, while the continuous object pose is estimated using analytical gradients for efficient optimization. By continuously updating the parameters of the manipulation potential, the framework dynamically reshapes the induced EM to guide online trajectory replanning and implement uncertainty-aware impedance control, thereby closing the perception-action loop. The system is validated through simulation and over 260 real-world screw-loosening trials. Experimental results demonstrate robust identification and manipulation success in standard scenarios while maintaining accurate tracking. Furthermore, ablation studies confirm that haptic SLAM and uncertainty-aware stiffness modulation outperform fixed impedance baselines, effectively preventing jamming during tight tolerance interactions.