Predicting cognitive load in immersive driving scenarios with a hybrid CNN-RNN model

TL;DR

This study introduces a hybrid CNN-RNN model that accurately predicts three levels of cognitive load in drivers using multimodal data during challenging driving conditions, advancing traffic safety research.

Contribution

It is the first to combine fNIRS, eye-tracking, and driving behavior data with a hybrid neural network for multi-level cognitive load prediction in real traffic scenarios.

Findings

Model achieves up to 99.99% accuracy with physiological data.

Model improves accuracy to 92.02% using driving behavior data.

Effective prediction under nighttime and rainy conditions.

Abstract

One debatable issue in traffic safety research is that cognitive load from sec-ondary tasks reduces primary task performance, such as driving. Although physiological signals have been extensively used in driving-related research to assess cognitive load, only a few studies have specifically focused on high cognitive load scenarios. Most existing studies tend to examine moderate or low levels of cognitive load In this study, we adopted an auditory version of the n-back task of three levels as a cognitively loading secondary task while driving in a driving simulator. During the simultaneous execution of driving and the n-back task, we recorded fNIRS, eye-tracking, and driving behavior data to predict cognitive load at three different levels. To the best of our knowledge, this combination of data sources has never been used before. Un-like most previous studies that utilize binary classification of cognitive load and driving in conditions without traffic, our study involved three levels of cognitive load, with drivers operating in normal traffic conditions under low visibility, specifically during nighttime and rainy weather. We proposed a hybrid neural network combining a 1D Convolutional Neural Network and a Recurrent Neural Network to predict cognitive load. Our experimental re-sults demonstrate that the proposed model, with fewer parameters, increases accuracy from 99.82% to 99.99% using physiological data, and from 87.26% to 92.02% using driving behavior data alone. This significant improvement highlights the effectiveness of our hybrid neural network in accurately pre-dicting cognitive load during driving under challenging conditions.