# Annotating Covert Hazardous Driving Scenarios Online: Utilizing Drivers' Electroencephalography (EEG) Signals

## TL;DR

This study explores a novel EEG-based method to implicitly label hazardous driving scenarios, potentially reducing manual annotation efforts and bias in creating driving databases for autonomous systems.

## Contribution

It introduces a passive EEG-based technique to detect covert hazards in driving scenarios, outperforming explicit danger reports and enabling more efficient data annotation.

## Key findings

- EEG signals are more sensitive to hazards than explicit reports.
- TSAI successfully classifies EEG signals for overt and covert hazards.
- Participants only reported overt hazards, but EEG detected covert hazards.

## Abstract

As autonomous driving systems prevail, it is becoming increasingly critical that the systems learn from databases containing fine-grained driving scenarios. Most databases currently available are human-annotated; they are expensive, time-consuming, and subject to behavioral biases. In this paper, we provide initial evidence supporting a novel technique utilizing drivers' electroencephalography (EEG) signals to implicitly label hazardous driving scenarios while passively viewing recordings of real-road driving, thus sparing the need for manual annotation and avoiding human annotators' behavioral biases during explicit report. We conducted an EEG experiment using real-life and animated recordings of driving scenarios and asked participants to report danger explicitly whenever necessary. Behavioral results showed the participants tended to report danger only when overt hazards (e.g., a vehicle or a pedestrian appearing unexpectedly from behind an occlusion) were in view. By contrast, their EEG signals were enhanced at the sight of both an overt hazard and a covert hazard (e.g., an occlusion signalling possible appearance of a vehicle or a pedestrian from behind). Thus, EEG signals were more sensitive to driving hazards than explicit reports. Further, the Time-Series AI (TSAI) successfully classified EEG signals corresponding to overt and covert hazards. We discuss future steps necessary to materialize the technique in real life.