FusionNet: 3D Object Classification Using Multiple Data Representations

TL;DR

FusionNet introduces a novel approach combining volumetric and pixel data representations with new volumetric CNN architectures to improve 3D object classification accuracy, surpassing existing methods on the Princeton ModelNet challenge.

Contribution

The paper presents a new multi-representation framework and volumetric CNN architectures that effectively fuse 3D voxel and 2D pixel data for enhanced object recognition.

Findings

Significantly improved classification accuracy over single-representation methods.

Effective fusion of volumetric and pixel data enhances feature learning.

New volumetric CNN architectures outperform existing models.

Abstract

High-quality 3D object recognition is an important component of many vision and robotics systems. We tackle the object recognition problem using two data representations, to achieve leading results on the Princeton ModelNet challenge. The two representations: 1. Volumetric representation: the 3D object is discretized spatially as binary voxels - $1$ if the voxel is occupied and $0$ otherwise. 2. Pixel representation: the 3D object is represented as a set of projected 2D pixel images. Current leading submissions to the ModelNet Challenge use Convolutional Neural Networks (CNNs) on pixel representations. However, we diverge from this trend and additionally, use Volumetric CNNs to bridge the gap between the efficiency of the above two representations. We combine both representations and exploit them to learn new features, which yield a significantly better classifier than using either of the representations in isolation. To do this, we introduce new Volumetric CNN (V-CNN) architectures.