Affordance Labeling and Exploration: A Manifold-Based Approach

TL;DR

This paper introduces a manifold-based approach to affordance labeling using pre-trained networks, achieving high accuracy and discovering new affordance labels without modifying network architectures.

Contribution

It proposes novel manifold curvature and subspace clustering methods for affordance recognition leveraging pre-trained networks without additional training or layer modifications.

Findings

Manifold curvature method exceeds 95% accuracy on nine networks.

Both methods identify affordance labels not present in ground truth.

Approach enables affordance exploration without retraining or network modification.

Abstract

The advancement in computing power has significantly reduced the training times for deep learning, fostering the rapid development of networks designed for object recognition. However, the exploration of object utility, which is the affordance of the object, as opposed to object recognition, has received comparatively less attention. This work focuses on the problem of exploration of object affordances using existing networks trained on the object classification dataset. While pre-trained networks have proven to be instrumental in transfer learning for classification tasks, this work diverges from conventional object classification methods. Instead, it employs pre-trained networks to discern affordance labels without the need for specialized layers, abstaining from modifying the final layers through the addition of classification layers. To facilitate the determination of affordance labels without such modifications, two approaches, i.e. subspace clustering and manifold curvature methods are tested. These methods offer a distinct perspective on affordance label recognition. Especially, manifold curvature method has been successfully tested with nine distinct pre-trained networks, each achieving an accuracy exceeding 95%. Moreover, it is observed that manifold curvature and subspace clustering methods explore affordance labels that are not marked in the ground truth, but object affords in various cases.