Anytime-Lidar: Deadline-aware 3D Object Detection

TL;DR

This paper introduces Anytime-Lidar, a scheduling framework that makes 3D object detection more flexible and efficient by dynamically balancing accuracy and computational deadlines in neural network components.

Contribution

It presents a novel deadline-aware scheduling algorithm for 3D object detection networks, improving accuracy under time constraints by selectively skipping neural network components.

Findings

Significantly improves detection accuracy under various deadline constraints.

Effectively balances computational time and detection accuracy.

Demonstrates effectiveness on Jetson Xavier AGX with nuScenes dataset.

Abstract

In this work, we present a novel scheduling framework enabling anytime perception for deep neural network (DNN) based 3D object detection pipelines. We focus on computationally expensive region proposal network (RPN) and per-category multi-head detector components, which are common in 3D object detection pipelines, and make them deadline-aware. We propose a scheduling algorithm, which intelligently selects the subset of the components to make effective time and accuracy trade-off on the fly. We minimize accuracy loss of skipping some of the neural network sub-components by projecting previously detected objects onto the current scene through estimations. We apply our approach to a state-of-art 3D object detection network, PointPillars, and evaluate its performance on Jetson Xavier AGX using nuScenes dataset. Compared to the baselines, our approach significantly improve the network's accuracy under various deadline constraints.