A Review of Reward Functions for Reinforcement Learning in the context of Autonomous Driving

TL;DR

This paper reviews reward functions used in reinforcement learning for autonomous driving, highlighting their limitations, lack of standardization, and proposing future directions for more effective, context-aware reward design.

Contribution

It systematically assesses existing reward formulations, categorizes objectives, discusses limitations, and suggests future research directions including validation frameworks and structured rewards.

Findings

Reward functions often lack standardization.

Current reward formulations inadequately address driving context.

Proposed future work includes context-aware and conflict-resolving rewards.

Abstract

Reinforcement learning has emerged as an important approach for autonomous driving. A reward function is used in reinforcement learning to establish the learned skill objectives and guide the agent toward the optimal policy. Since autonomous driving is a complex domain with partly conflicting objectives with varying degrees of priority, developing a suitable reward function represents a fundamental challenge. This paper aims to highlight the gap in such function design by assessing different proposed formulations in the literature and dividing individual objectives into Safety, Comfort, Progress, and Traffic Rules compliance categories. Additionally, the limitations of the reviewed reward functions are discussed, such as objectives aggregation and indifference to driving context. Furthermore, the reward categories are frequently inadequately formulated and lack standardization. This paper concludes by proposing future research that potentially addresses the observed shortcomings in rewards, including a reward validation framework and structured rewards that are context-aware and able to resolve conflicts.