Bio-Inspired 3D Affordance Understanding from Single Image with Neural Radiance Field for Enhanced Embodied Intelligence

TL;DR

This paper introduces AFF-NeRF, a method that uses neural radiance fields to generate 3D affordance models from single images, improving robotic manipulation of unseen objects.

Contribution

The novel AFF-NeRF method combines deep learning and neural radiance fields to generate 3D affordance models for novel objects from a single image.

Findings

AFF-NeRF outperforms baseline methods in affordance generation for unseen objects.

3D affordance models from AFF-NeRF lead to more stable robotic grasps.

The method adapts to various homogeneous objects without additional training.

Abstract

Affordance understanding means identifying possible operable parts of objects, which is crucial in achieving accurate robotic manipulation. Although homogeneous objects for grasping have various shapes, they always share a similar affordance distribution. Based on this fact, we propose AFF-NeRF to address the problem of affordance generation for homogeneous objects inspired by human cognitive processes. Our method employs deep residual networks to extract the shape and appearance features of various objects, enabling it to adapt to various homogeneous objects. These features are then integrated into our extended neural radiance fields, named AFF-NeRF, to generate 3D affordance models for unseen objects using a single image. Our experimental results demonstrate that our approach outperforms baseline methods in the affordance generation of unseen views on novel objects without additional training. Additionally, more stable grasps can be obtained by employing 3D affordance models generated by our method in the grasp generation algorithm.

Figures