Mixed Precision PointPillars for Efficient 3D Object Detection with TensorRT

TL;DR

This paper introduces a mixed precision quantization framework for PointPillars 3D object detection, significantly reducing latency with minimal performance loss by selectively using INT8 and floating point precision.

Contribution

The authors propose a novel mixed precision quantization method for PointPillars that improves inference speed while maintaining high accuracy, using a layer sensitivity search and outlier handling techniques.

Findings

Mixed precision models reduce latency by up to 2.538 times.

Selective layer quantization maintains competitive accuracy.

Outlier handling improves post-training quantization performance.

Abstract

LIDAR 3D object detection is one of the important tasks for autonomous vehicles. Ensuring that this task operates in real-time is crucial. Toward this, model quantization can be used to accelerate the runtime. However, directly applying model quantization often leads to performance degradation due to LIDAR's wide numerical distributions and extreme outliers. To address the wide numerical distribution, we proposed a mixed precision framework designed for PointPillars. Our framework first searches for sensitive layers with post-training quantization (PTQ) by quantizing one layer at a time to 8-bit integer (INT8) and evaluating each model for average precision (AP). The top-k most sensitive layers are assigned as floating point (FP). Combinations of these layers are greedily searched to produce candidate mixed precision models, which are finalized with either PTQ or quantization-aware training (QAT). Furthermore, to handle outliers, we observe that using a very small number of calibration data reduces the likelihood of encountering outliers, thereby improving PTQ performance. Our methods provides mixed precision models without training in the PTQ pipeline, while our QAT pipeline achieves the performance competitive to FP models. With TensorRT deployment, our mixed precision models offer less latency by up to 2.538 times compared to FP32 models.