Transformed Multi-view 3D Shape Features with Contrastive Learning

TL;DR

This paper explores the use of Vision Transformers combined with contrastive learning techniques to improve 3D shape feature representation, achieving high accuracy with less labeled data in multi-view 3D analysis.

Contribution

It introduces a novel approach that integrates ViTs with contrastive learning for 3D shape understanding, demonstrating superior performance over traditional CNN-based methods.

Findings

Achieved 90.6% accuracy on ModelNet10.

ViTs effectively capture global shape semantics.

Contrastive learning refines local discriminative features.

Abstract

This paper addresses the challenges in representation learning of 3D shape features by investigating state-of-the-art backbones paired with both contrastive supervised and self-supervised learning objectives. Computer vision methods struggle with recognizing 3D objects from 2D images, often requiring extensive labeled data and relying on Convolutional Neural Networks (CNNs) that may overlook crucial shape relationships. Our work demonstrates that Vision Transformers (ViTs) based architectures, when paired with modern contrastive objectives, achieve promising results in multi-view 3D analysis on our downstream tasks, unifying contrastive and 3D shape understanding pipelines. For example, supervised contrastive losses reached about 90.6% accuracy on ModelNet10. The use of ViTs and contrastive learning, leveraging ViTs' ability to understand overall shapes and contrastive learning's effectiveness, overcomes the need for extensive labeled data and the limitations of CNNs in capturing crucial shape relationships. The success stems from capturing global shape semantics via ViTs and refining local discriminative features through contrastive optimization. Importantly, our approach is empirical, as it is grounded on extensive experimental evaluation to validate the effectiveness of combining ViTs with contrastive objectives for 3D representation learning.