The Integration of Prediction and Planning in Deep Learning Automated Driving Systems: A Review

TL;DR

This review analyzes how recent deep learning automated driving systems integrate prediction and planning, emphasizing their architectures, behavioral modeling, and future research directions to enhance safety and efficiency.

Contribution

It provides a comprehensive overview of integration principles in deep learning-based automated driving, highlighting recent advances, challenges, and future research opportunities.

Findings

Integration of prediction and planning improves traffic interaction modeling.

Different architectural approaches have distinct strengths and limitations.

Identifies key research gaps and future challenges in the field.

Abstract

Automated driving has the potential to revolutionize personal, public, and freight mobility. Beside accurately perceiving the environment, automated vehicles must plan a safe, comfortable, and efficient motion trajectory. To promote safety and progress, many works rely on modules that predict the future motion of surrounding traffic. Modular automated driving systems commonly handle prediction and planning as sequential, separate tasks. While this accounts for the influence of surrounding traffic on the ego vehicle, it fails to anticipate the reactions of traffic participants to the ego vehicle's behavior. Recent methods increasingly integrate prediction and planning in a joint or interdependent step to model bidirectional interactions. To date, a comprehensive overview of different integration principles is lacking. We systematically review state-of-the-art deep learning-based planning systems, and focus on how they integrate prediction. Different facets of the integration ranging from system architecture to high-level behavioral aspects are considered and related to each other. Moreover, we discuss the implications, strengths, and limitations of different integration principles. By pointing out research gaps, describing relevant future challenges, and highlighting trends in the research field, we identify promising directions for future research.