Multi-Task Network Pruning and Embedded Optimization for Real-time Deployment in ADAS

TL;DR

This paper presents a multi-task CNN pruning and optimization approach tailored for real-time deployment in automotive embedded systems, achieving significant compression and efficiency improvements without sacrificing detection performance.

Contribution

It introduces a novel multi-task network architecture combined with pruning and embedded optimization techniques for automotive perception tasks.

Findings

Reduced runtime and memory usage by a factor of 2

Achieved real-time processing on a low-power SoC

Maintained detection performance despite compression

Abstract

Camera-based Deep Learning algorithms are increasingly needed for perception in Automated Driving systems. However, constraints from the automotive industry challenge the deployment of CNNs by imposing embedded systems with limited computational resources. In this paper, we propose an approach to embed a multi-task CNN network under such conditions on a commercial prototype platform, i.e. a low power System on Chip (SoC) processing four surround-view fisheye cameras at 10 FPS. The first focus is on designing an efficient and compact multi-task network architecture. Secondly, a pruning method is applied to compress the CNN, helping to reduce the runtime and memory usage by a factor of 2 without lowering the performances significantly. Finally, several embedded optimization techniques such as mixed-quantization format usage and efficient data transfers between different memory areas are proposed to ensure real-time execution and avoid bandwidth bottlenecks. The approach is evaluated on the hardware platform, considering embedded detection performances, runtime and memory bandwidth. Unlike most works from the literature that focus on classification task, we aim here to study the effect of pruning and quantization on a compact multi-task network with object detection, semantic segmentation and soiling detection tasks.