Grounded Affordance from Exocentric View
Hongchen Luo, Wei Zhai, Jing Zhang, Yang Cao, Dacheng Tao

TL;DR
This paper introduces a novel framework for affordance grounding from exocentric views, transferring knowledge to egocentric images to improve understanding of object action possibilities, with a new dataset and superior performance.
Contribution
It proposes a cross-view affordance knowledge transfer method and introduces the AGD20K dataset for affordance grounding tasks.
Findings
Outperforms existing models on objective metrics
Enhances perception of affordance regions
Constructed a large-scale affordance dataset AGD20K
Abstract
Affordance grounding aims to locate objects' "action possibilities" regions, which is an essential step toward embodied intelligence. Due to the diversity of interactive affordance, the uniqueness of different individuals leads to diverse interactions, which makes it difficult to establish an explicit link between object parts and affordance labels. Human has the ability that transforms the various exocentric interactions into invariant egocentric affordance to counter the impact of interactive diversity. To empower an agent with such ability, this paper proposes a task of affordance grounding from exocentric view, i.e., given exocentric human-object interaction and egocentric object images, learning the affordance knowledge of the object and transferring it to the egocentric image using only the affordance label as supervision. However, there is some "interaction bias" between…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsRobot Manipulation and Learning · Human Pose and Action Recognition · Advanced Vision and Imaging
