Autonomous Vehicles that Alert Humans to Take-Over Controls: Modeling with Real-World Data

TL;DR

This paper develops models to predict when a human driver should take control from an autonomous vehicle, using real-world data and computer vision to improve safety and transition timing.

Contribution

It introduces a large-scale real-world dataset of control transitions and develops sequential models for predicting take-over times based on driver state features.

Findings

Models predict take-over times accurately in real-world scenarios

Sequential features improve prediction quality

Dataset enables future research on driver-vehicle interaction

Abstract

With increasing automation in passenger vehicles, the study of safe and smooth occupant-vehicle interaction and control transitions is key. In this study, we focus on the development of contextual, semantically meaningful representations of the driver state, which can then be used to determine the appropriate timing and conditions for transfer of control between driver and vehicle. To this end, we conduct a large-scale real-world controlled data study where participants are instructed to take-over control from an autonomous agent under different driving conditions while engaged in a variety of distracting activities. These take-over events are captured using multiple driver-facing cameras, which when labelled result in a dataset of control transitions and their corresponding take-over times (TOTs). We then develop and train TOT models that operate sequentially on mid to high-level features produced by computer vision algorithms operating on different driver-facing camera views. The proposed TOT model produces continuous predictions of take-over times without delay, and shows promising qualitative and quantitative results in complex real-world scenarios.