Driving Scene Perception Network: Real-time Joint Detection, Depth Estimation and Semantic Segmentation

TL;DR

This paper presents DSPNet, a unified neural network for real-time object detection, depth estimation, and semantic segmentation in driving scenes, optimizing accuracy and efficiency for autonomous driving applications.

Contribution

Introduction of DSPNet, a shared architecture that performs multiple perception tasks simultaneously with high efficiency and improved accuracy over single-task models.

Findings

Achieves 14.0 fps on GTX 1080 with 1024x512 images.

Uses less than 850 MiB GPU memory.

Improves precision and efficiency compared to separate models.

Abstract

As the demand for enabling high-level autonomous driving has increased in recent years and visual perception is one of the critical features to enable fully autonomous driving, in this paper, we introduce an efficient approach for simultaneous object detection, depth estimation and pixel-level semantic segmentation using a shared convolutional architecture. The proposed network model, which we named Driving Scene Perception Network (DSPNet), uses multi-level feature maps and multi-task learning to improve the accuracy and efficiency of object detection, depth estimation and image segmentation tasks from a single input image. Hence, the resulting network model uses less than 850 MiB of GPU memory and achieves 14.0 fps on NVIDIA GeForce GTX 1080 with a 1024x512 input image, and both precision and efficiency have been improved over combination of single tasks.