Push to know! -- Visuo-Tactile based Active Object Parameter Inference with Dual Differentiable Filtering

TL;DR

This paper introduces a novel active dual differentiable filtering framework that enables robots to estimate physical object properties through visuo-tactile sensing during non-prehensile pushes, improving interaction with novel objects.

Contribution

The work presents a new active filtering approach that learns object-robot interactions and efficiently infers object parameters using combined vision and tactile data during push actions.

Findings

Outperforms state-of-the-art methods in simulation and real robots.

Effectively estimates shape, mass, and friction of novel objects.

Enables adaptive exploration through active push strategies.

Abstract

For robotic systems to interact with objects in dynamic environments, it is essential to perceive the physical properties of the objects such as shape, friction coefficient, mass, center of mass, and inertia. This not only eases selecting manipulation action but also ensures the task is performed as desired. However, estimating the physical properties of especially novel objects is a challenging problem, using either vision or tactile sensing. In this work, we propose a novel framework to estimate key object parameters using non-prehensile manipulation using vision and tactile sensing. Our proposed active dual differentiable filtering (ADDF) approach as part of our framework learns the object-robot interaction during non-prehensile object push to infer the object's parameters. Our proposed method enables the robotic system to employ vision and tactile information to interactively explore a novel object via non-prehensile object push. The novel proposed N-step active formulation within the differentiable filtering facilitates efficient learning of the object-robot interaction model and during inference by selecting the next best exploratory push actions (where to push? and how to push?). We extensively evaluated our framework in simulation and real-robotic scenarios, yielding superior performance to the state-of-the-art baseline.