CompassAD: Intent-Driven 3D Affordance Grounding in Functionally Competing Objects

TL;DR

This paper introduces a new benchmark and framework for intent-driven 3D affordance grounding in scenes with multiple similar objects, enabling robots to identify the correct object based on natural language intent.

Contribution

It formalizes the challenging problem of confusable affordance grounding, creates the CompassAD benchmark, and proposes CompassNet with novel modules for improved accuracy.

Findings

State-of-the-art performance on the CompassAD benchmark.

Effective transfer of the method to real-world robotic grasping.

Proposed modules prevent semantic leakage and improve discrimination.

Abstract

When told to "cut the cake," a robot must choose the knife over nearby scissors, despite both objects affording the same cutting function. In real-world scenes, multiple objects may share identical affordances, yet only one is appropriate under the given task context. We call such cases confusing pairs. However, existing 3D affordance methods largely sidestep this challenge by evaluating isolated single objects, often with explicit category names provided in the query. We formalize Intent-Driven Confusable Affordance Grounding, a new 3D affordance setting that requires predicting a per-point affordance mask on the correct object within a multi-object point cloud, conditioned on implicit natural language intent. To study this problem, we construct CompassAD, the first benchmark centered on implicit intent in confusing multi-object compositions. It comprises 30 confusing object pairs spanning 16 affordance types, 6,422 compositions, and 88K+ query-answer pairs. Furthermore, we propose CompassNet, a framework that incorporates two dedicated modules tailored to this task. Instance-bounded Cross Injection (ICI) constrains language-geometry alignment within object boundaries to prevent cross-object semantic leakage. Bi-level Contrastive Refinement (BCR) enforces discrimination at both geometric-group and point levels, sharpening distinctions between target and confusable surfaces. Extensive experiments demonstrate state-of-the-art results on both seen and unseen queries, and deployment on a robotic manipulator confirms effective transfer to real-world grasping in confusing multi-object compositions.