# To New Beginnings: A Survey of Unified Perception in Autonomous Vehicle Software

## TL;DR

This survey reviews unified perception in autonomous vehicles, which integrates detection, tracking, and prediction tasks into a shared architecture to improve robustness, efficiency, and interpretability.

## Contribution

It introduces a comprehensive taxonomy and framework for unified perception, categorizing methods and guiding future research in the field.

## Key findings

- Defines three paradigms: Early, Late, and Full Unified Perception.
- Provides systematic review of existing methods, architectures, and datasets.
- Highlights open challenges and future research directions.

## Abstract

Autonomous vehicle perception typically relies on modular pipelines that decompose the task into detection, tracking, and prediction. While interpretable, these pipelines suffer from error accumulation and limited inter-task synergy. Unified perception has emerged as a promising paradigm that integrates these sub-tasks within a shared architecture, potentially improving robustness, contextual reasoning, and efficiency while retaining interpretable outputs. In this survey, we provide a comprehensive overview of unified perception, introducing a holistic and systemic taxonomy that categorizes methods along task integration, tracking formulation, and representation flow. We define three paradigms -Early, Late, and Full Unified Perception- and systematically review existing methods, their architectures, training strategies, datasets used, and open-source availability, while highlighting future research directions. This work establishes the first comprehensive framework for understanding and advancing unified perception, consolidates fragmented efforts, and guides future research toward more robust, generalizable, and interpretable perception.