Explainable Object-induced Action Decision for Autonomous Vehicles

TL;DR

This paper introduces a human-inspired paradigm for autonomous driving that emphasizes understanding and explaining object-induced actions, improving decision accuracy through joint action and explanation modeling.

Contribution

It proposes a novel multi-task CNN framework that jointly predicts vehicle actions and explanations based on action-inducing objects, extending the BDD100K dataset.

Findings

Explanation requirement enhances object recognition accuracy.

Joint modeling improves action prediction performance.

The approach bridges end-to-end and pipelined methods with explainability.

Abstract

A new paradigm is proposed for autonomous driving. The new paradigm lies between the end-to-end and pipelined approaches, and is inspired by how humans solve the problem. While it relies on scene understanding, the latter only considers objects that could originate hazard. These are denoted as action-inducing, since changes in their state should trigger vehicle actions. They also define a set of explanations for these actions, which should be produced jointly with the latter. An extension of the BDD100K dataset, annotated for a set of 4 actions and 21 explanations, is proposed. A new multi-task formulation of the problem, which optimizes the accuracy of both action commands and explanations, is then introduced. A CNN architecture is finally proposed to solve this problem, by combining reasoning about action inducing objects and global scene context. Experimental results show that the requirement of explanations improves the recognition of action-inducing objects, which in turn leads to better action predictions.