Affordance Transfer Across Object Instances via Semantically Anchored Functional Map

TL;DR

This paper introduces SemFM, a framework that transfers affordances across different objects using semantic anchors and functional maps, enabling robots to generalize learned interactions from a single visual demonstration.

Contribution

The paper presents a novel semantic anchored functional map method for transferring affordances across diverse objects from minimal demonstrations.

Findings

Accurate affordance transfer demonstrated on synthetic and real-world objects.

Method achieves lightweight and interpretable correspondence between objects.

Effective in practical robotic perception and manipulation tasks.

Abstract

Traditional learning from demonstration (LfD) generally demands a cumbersome collection of physical demonstrations, which can be time-consuming and challenging to scale. Recent advances show that robots can instead learn from human videos by extracting interaction cues without direct robot involvement. However, a fundamental challenge remains: how to generalize demonstrated interactions across different object instances that share similar functionality but vary significantly in geometry. In this work, we propose \emph{Semantic Anchored Functional Maps} (SemFM), a framework for transferring affordances across objects from a single visual demonstration. Starting from a coarse mesh reconstructed from an image, our method identifies semantically corresponding functional regions between objects, selects mutually exclusive semantic anchors, and propagates these constraints over the surface using a functional map to obtain a dense, semantically consistent correspondence. This enables demonstrated interaction regions to be transferred across geometrically diverse objects in a lightweight and interpretable manner. Experiments on synthetic object categories and real-world robotic manipulation tasks show that our approach enables accurate affordance transfer with modest computational cost, making it well-suited for practical robotic perception-to-action pipelines.