Predicting decision-making in the future: Human versus Machine

TL;DR

This study compares human and deep neural network predictions of decision-making in a driving simulation, revealing that DNNs outperform humans early on but perform equally later, with insights into their processing differences.

Contribution

The paper demonstrates how DNNs can predict human decision-making in dynamic scenarios and explores their temporal processing similarities and differences with humans.

Findings

DNNs outperform humans in early prediction stages.

Humans and DNNs have similar performance closer to the event.

Visualizations reveal differences in temporal processing.

Abstract

Deep neural networks (DNNs) have become remarkably successful in data prediction, and have even been used to predict future actions based on limited input. This raises the question: do these systems actually "understand" the event similar to humans? Here, we address this issue using videos taken from an accident situation in a driving simulation. In this situation, drivers had to choose between crashing into a suddenly-appeared obstacle or steering their car off a previously indicated cliff. We compared how well humans and a DNN predicted this decision as a function of time before the event. The DNN outperformed humans for early time-points, but had an equal performance for later time-points. Interestingly, spatio-temporal image manipulations and Grad-CAM visualizations uncovered some expected behavior, but also highlighted potential differences in temporal processing for the DNN.