Multi Voxel-Point Neurons Convolution (MVPConv) for Fast and Accurate 3D Deep Learning

TL;DR

MVPConv is a novel 3D deep learning convolution that combines voxel and point-based features, significantly improving accuracy and speed across various 3D tasks and datasets.

Contribution

The paper introduces MVPConv, a new convolutional module that integrates voxel and point-based features, enabling versatile and efficient 3D deep learning across multiple tasks.

Findings

Improves PointNet accuracy by up to 36%.

Achieves up to 34 times speedup over voxel-based models.

Outperforms state-of-the-art point-based models with up to 8 times speedup.

Abstract

We present a new convolutional neural network, called Multi Voxel-Point Neurons Convolution (MVPConv), for fast and accurate 3D deep learning. The previous works adopt either individual point-based features or local-neighboring voxel-based features to process 3D model, which limits the performance of models due to the inefficient computation. Moreover, most of the existing 3D deep learning frameworks aim at solving one specific task, and only a few of them can handle a variety of tasks. Integrating both the advantages of the voxel and point-based methods, the proposed MVPConv can effectively increase the neighboring collection between point-based features and also promote the independence among voxel-based features. Simply replacing the corresponding convolution module with MVPConv, we show that MVPConv can fit in different backbones to solve a wide range of 3D tasks. Extensive experiments on benchmark datasets such as ShapeNet Part, S3DIS and KITTI for various tasks show that MVPConv improves the accuracy of the backbone (PointNet) by up to 36%, and achieves higher accuracy than the voxel-based model with up to 34 times speedup. In addition, MVPConv also outperforms the state-of-the-art point-based models with up to 8 times speedup. Notably, our MVPConv achieves better accuracy than the newest point-voxel-based model PVCNN (a model more efficient than PointNet) with lower latency.