One-Shot Affordance Detection

TL;DR

This paper introduces a novel one-shot affordance detection network that identifies common action possibilities in images using a support example, enabling robots to perceive unseen affordances effectively.

Contribution

The paper proposes the OS-AD network and a new PAD dataset, advancing one-shot affordance detection with purpose estimation and transfer capabilities.

Findings

OS-AD outperforms previous methods in accuracy and visual quality.

The dataset enables robust training and evaluation of affordance detection models.

Experimental results validate the effectiveness of the proposed approach.

Abstract

Affordance detection refers to identifying the potential action possibilities of objects in an image, which is an important ability for robot perception and manipulation. To empower robots with this ability in unseen scenarios, we consider the challenging one-shot affordance detection problem in this paper, i.e., given a support image that depicts the action purpose, all objects in a scene with the common affordance should be detected. To this end, we devise a One-Shot Affordance Detection (OS-AD) network that firstly estimates the purpose and then transfers it to help detect the common affordance from all candidate images. Through collaboration learning, OS-AD can capture the common characteristics between objects having the same underlying affordance and learn a good adaptation capability for perceiving unseen affordances. Besides, we build a Purpose-driven Affordance Dataset (PAD) by collecting and labeling 4k images from 31 affordance and 72 object categories. Experimental results demonstrate the superiority of our model over previous representative ones in terms of both objective metrics and visual quality. The benchmark suite is at ProjectPage.