Affordance detection with Dynamic-Tree Capsule Networks

TL;DR

This paper introduces a novel dynamic-tree capsule network for affordance detection in 3D point clouds, significantly improving viewpoint invariance and parts-to-whole understanding over traditional CNN-based methods, especially for unseen objects.

Contribution

The first affordance detection network based on dynamic tree-structured capsules for sparse 3D point clouds, enhancing robustness to novel objects and viewpoints.

Findings

Outperforms state-of-the-art models in viewpoint invariance

Achieves better parts-segmentation of new object instances

Superior in grasping unseen objects

Abstract

Affordance detection from visual input is a fundamental step in autonomous robotic manipulation. Existing solutions to the problem of affordance detection rely on convolutional neural networks. However, these networks do not consider the spatial arrangement of the input data and miss parts-to-whole relationships. Therefore, they fall short when confronted with novel, previously unseen object instances or new viewpoints. One solution to overcome such limitations can be to resort to capsule networks. In this paper, we introduce the first affordance detection network based on dynamic tree-structured capsules for sparse 3D point clouds. We show that our capsule-based network outperforms current state-of-the-art models on viewpoint invariance and parts-segmentation of new object instances through a novel dataset we only used for evaluation and it is publicly available from github.com/gipfelen/DTCG-Net. In the experimental evaluation we will show that our algorithm is superior to current affordance detection methods when faced with grasping previously unseen objects thanks to our Capsule Network enforcing a parts-to-whole representation.