Achieving Real-Time LiDAR 3D Object Detection on a Mobile Device

TL;DR

This paper presents a novel compiler-aware framework that combines network enhancement and pruning via reinforcement learning to enable real-time 3D LiDAR object detection on resource-constrained mobile devices, such as smartphones.

Contribution

It introduces a generator RNN that automates network optimization for real-time detection without human intervention, tailored for edge devices.

Findings

Real-time 3D detection achieved on Samsung Galaxy S20

Competitive detection performance maintained

Automated network optimization without human expertise

Abstract

3D object detection is an important task, especially in the autonomous driving application domain. However, it is challenging to support the real-time performance with the limited computation and memory resources on edge-computing devices in self-driving cars. To achieve this, we propose a compiler-aware unified framework incorporating network enhancement and pruning search with the reinforcement learning techniques, to enable real-time inference of 3D object detection on the resource-limited edge-computing devices. Specifically, a generator Recurrent Neural Network (RNN) is employed to provide the unified scheme for both network enhancement and pruning search automatically, without human expertise and assistance. And the evaluated performance of the unified schemes can be fed back to train the generator RNN. The experimental results demonstrate that the proposed framework firstly achieves real-time 3D object detection on mobile devices (Samsung Galaxy S20 phone) with competitive detection performance.