Compressing Multi-Task Model for Autonomous Driving via Pruning and Knowledge Distillation

TL;DR

This paper presents a novel framework combining task-aware pruning and feature-level knowledge distillation to compress multi-task autonomous driving models, significantly reducing parameters while maintaining high performance and real-time speed.

Contribution

It introduces a safe pruning strategy with Taylor importance and gradient conflict penalty, along with a task head-agnostic distillation method for effective multi-task model compression.

Findings

Parameter reduction of 32.7% with negligible accuracy loss

Maintains real-time inference at 32.7 FPS

Achieves effective compression without significant performance degradation

Abstract

Autonomous driving systems rely on panoptic perception to jointly handle object detection, drivable area segmentation, and lane line segmentation. Although multi-task learning is an effective way to integrate these tasks, its increasing model parameters and complexity make deployment on on-board devices difficult. To address this challenge, we propose a multi-task model compression framework that combines task-aware safe pruning with feature-level knowledge distillation. Our safe pruning strategy integrates Taylor-based channel importance with gradient conflict penalty to keep important channels while removing redundant and conflicting channels. To mitigate performance degradation after pruning, we further design a task head-agnostic distillation method that transfers intermediate backbone and encoder features from a teacher to a student model as guidance. Experiments on the BDD100K dataset demonstrate that our compressed model achieves a 32.7% reduction in parameters while segmentation performance shows negligible accuracy loss and only a minor decrease in detection (-1.2% for Recall and -1.8% for mAP50) compared to the teacher. The compressed model still runs at 32.7 FPS in real-time. These results show that combining pruning and knowledge distillation provides an effective compression solution for multi-task panoptic perception.