Spatial-temporal Fusion Convolutional Neural Network for Simulated Driving Behavior Recognition

TL;DR

This paper introduces a two-stream CNN framework for recognizing abnormal driving behaviors from video data, combining appearance and motion cues to improve accuracy in simulated environments.

Contribution

It proposes a novel spatial-temporal fusion strategy within a two-stream CNN for driving behavior recognition, validated on a new simulated dataset.

Findings

Achieved improved recognition accuracy over existing methods.

Validated effectiveness of spatial-temporal fusion in driving behavior analysis.

Created a new dataset with 1237 videos for behavior recognition.

Abstract

Abnormal driving behaviour is one of the leading cause of terrible traffic accidents endangering human life. Therefore, study on driving behaviour surveillance has become essential to traffic security and public management. In this paper, we conduct this promising research and employ a two stream CNN framework for video-based driving behaviour recognition, in which spatial stream CNN captures appearance information from still frames, whilst temporal stream CNN captures motion information with pre-computed optical flow displacement between a few adjacent video frames. We investigate different spatial-temporal fusion strategies to combine the intra frame static clues and inter frame dynamic clues for final behaviour recognition. So as to validate the effectiveness of the designed spatial-temporal deep learning based model, we create a simulated driving behaviour dataset, containing 1237 videos with 6 different driving behavior for recognition. Experiment result shows that our proposed method obtains noticeable performance improvements compared to the existing methods.