LidarNAS: Unifying and Searching Neural Architectures for 3D Point Clouds

TL;DR

This paper introduces LidarNAS, a unified framework for neural architecture design in 3D point cloud understanding, enabling systematic exploration and outperforming existing models through neural architecture search.

Contribution

It proposes a modular, unified framework for 3D point cloud neural architectures and applies neural architecture search to improve performance on 3D detection tasks.

Findings

Outperforms state-of-the-art models on Waymo dataset

NAS discovers similar macro architectures for different object classes

Framework can reproduce and compare various existing architectures

Abstract

Developing neural models that accurately understand objects in 3D point clouds is essential for the success of robotics and autonomous driving. However, arguably due to the higher-dimensional nature of the data (as compared to images), existing neural architectures exhibit a large variety in their designs, including but not limited to the views considered, the format of the neural features, and the neural operations used. Lack of a unified framework and interpretation makes it hard to put these designs in perspective, as well as systematically explore new ones. In this paper, we begin by proposing a unified framework of such, with the key idea being factorizing the neural networks into a series of view transforms and neural layers. We demonstrate that this modular framework can reproduce a variety of existing works while allowing a fair comparison of backbone designs. Then, we show how this framework can easily materialize into a concrete neural architecture search (NAS) space, allowing a principled NAS-for-3D exploration. In performing evolutionary NAS on the 3D object detection task on the Waymo Open Dataset, not only do we outperform the state-of-the-art models, but also report the interesting finding that NAS tends to discover the same macro-level architecture concept for both the vehicle and pedestrian classes.