Hierarchical Abstraction Enables Human-Like 3D Object Recognition in Deep Learning Models

TL;DR

This paper investigates how hierarchical abstraction in deep learning models, especially point transformers, enables human-like recognition of 3D objects from sparse point cloud data, highlighting the importance of global shape representation.

Contribution

It demonstrates that hierarchical abstraction in point transformer models better mimics human 3D shape recognition than convolutional models, emphasizing the role of global shape processing.

Findings

Point transformer models outperform convolutional models in 3D recognition tasks.

Hierarchical abstraction mechanisms improve global shape understanding.

Humans perform consistently across various 3D recognition conditions.

Abstract

Both humans and deep learning models can recognize objects from 3D shapes depicted with sparse visual information, such as a set of points randomly sampled from the surfaces of 3D objects (termed a point cloud). Although deep learning models achieve human-like performance in recognizing objects from 3D shapes, it remains unclear whether these models develop 3D shape representations similar to those used by human vision for object recognition. We hypothesize that training with 3D shapes enables models to form representations of local geometric structures in 3D shapes. However, their representations of global 3D object shapes may be limited. We conducted two human experiments systematically manipulating point density and object orientation (Experiment 1), and local geometric structure (Experiment 2). Humans consistently performed well across all experimental conditions. We compared two types of deep learning models, one based on a convolutional neural network (DGCNN) and the other on visual transformers (point transformer), with human performance. We found that the point transformer model provided a better account of human performance than the convolution-based model. The advantage mainly results from the mechanism in the point transformer model that supports hierarchical abstraction of 3D shapes.