Brain informed transfer learning for categorizing construction hazards

TL;DR

This paper introduces a transfer learning approach that leverages EEG brain signals to pretrain CNNs for construction hazard classification, resulting in improved accuracy and insights into cognitive processing.

Contribution

It presents a novel method of using EEG data to pretrain CNNs, enhancing hazard recognition accuracy and integrating human brain signals into machine learning.

Findings

EEG-pretrained CNN outperforms randomly initialized networks by 9% accuracy.

Left frontal cortex activity correlates with higher hazard recognition performance.

The approach demonstrates the potential of brain-informed transfer learning for visual recognition tasks.

Abstract

A transfer learning paradigm is proposed for "knowledge" transfer between the human brain and convolutional neural network (CNN) for a construction hazard categorization task. Participants' brain activities are recorded using electroencephalogram (EEG) measurements when viewing the same images (target dataset) as the CNN. The CNN is pretrained on the EEG data and then fine-tuned on the construction scene images. The results reveal that the EEG-pretrained CNN achieves a 9 % higher accuracy compared with a network with same architecture but randomly initialized parameters on a three-class classification task. Brain activity from the left frontal cortex exhibits the highest performance gains, thus indicating high-level cognitive processing during hazard recognition. This work is a step toward improving machine learning algorithms by learning from human-brain signals recorded via a commercially available brain-computer interface. More generalized visual recognition systems can be effectively developed based on this approach of "keep human in the loop".