Coupled Particle Filters for Robust Affordance Estimation

TL;DR

This paper introduces a coupled recursive estimation method for robotic affordance detection that improves precision and robustness by integrating property-specific signals and bidirectional information exchange.

Contribution

It presents a novel coupled particle filter approach for disambiguating affordance signals, outperforming existing methods in accuracy and robustness in real-world scenarios.

Findings

Outperforms recent affordance estimators by over 200% in precision.

Achieves 70% success rate in real-world affordance tasks.

Robust under low light and cluttered conditions.

Abstract

Robotic affordance estimation is challenging due to visual, geometric, and semantic ambiguities in sensory input. We propose a method that disambiguates these signals using two coupled recursive estimators for sub-aspects of affordances: graspable and movable regions. Each estimator encodes property-specific regularities to reduce uncertainty, while their coupling enables bidirectional information exchange that focuses attention on regions where both agree, i.e., affordances. Evaluated on a real-world dataset, our method outperforms three recent affordance estimators (Where2Act, Hands-as-Probes, and HRP) by 308%, 245%, and 257% in precision, and remains robust under challenging conditions such as low light or cluttered environments. Furthermore, our method achieves a 70% success rate in our real-world evaluation. These results demonstrate that coupling complementary estimators yields precise, robust, and embodiment-appropriate affordance predictions.