A Survey on Deep Multi-Task Learning in Connected Autonomous Vehicles

TL;DR

This survey reviews the application of multi-task learning in connected autonomous vehicles, highlighting its potential to improve efficiency and resource use across perception, prediction, planning, and collaboration tasks.

Contribution

It is the first comprehensive review focusing on multi-task learning specifically within the context of connected autonomous vehicles.

Findings

MTL improves efficiency in CAV systems.

Current methods have notable limitations and research gaps.

Future directions include enhancing MTL techniques for CAVs.

Abstract

Connected autonomous vehicles (CAVs) must simultaneously perform multiple tasks, such as object detection, semantic segmentation, depth estimation, trajectory prediction, motion prediction, and behaviour prediction, to ensure safe and reliable navigation in complex environments. Vehicle-to-everything (V2X) communication enables cooperative driving among CAVs, thereby mitigating the limitations of individual sensors, reducing occlusions, and improving perception over long distances. Traditionally, these tasks are addressed using distinct models, which leads to high deployment costs, increased computational overhead, and challenges in achieving real-time performance. Multi-task learning (MTL) has recently emerged as a promising solution that enables the joint learning of multiple tasks within a single unified model. This offers improved efficiency and resource utilization. To the best of our knowledge, this survey is the first comprehensive review focused on MTL in the context of CAVs. We begin with an overview of CAVs and MTL to provide foundational background. We then explore the application of MTL across key functional modules, including perception, prediction, planning, control, and multi-agent collaboration. Finally, we discuss the strengths and limitations of existing methods, identify key research gaps, and provide directions for future research aimed at advancing MTL methodologies for CAV systems.