Rethinking Top Probability from Multi-view for Distracted Driver Behaviour Localization

TL;DR

This paper proposes a multi-view, self-supervised learning approach with a novel post-processing strategy to improve the localization of distracted driver behaviors in real-world video data, achieving competitive results in the AI City Challenge.

Contribution

It introduces a new multi-view ensemble and conditional post-processing method for more accurate behavior localization in driving videos, leveraging self-supervised recognition models.

Findings

Achieved sixth place on the AI City Challenge leaderboard.

Demonstrated improved localization accuracy with multi-view ensemble.

Validated effectiveness of the conditional post-processing operation.

Abstract

Naturalistic driving action localization task aims to recognize and comprehend human behaviors and actions from video data captured during real-world driving scenarios. Previous studies have shown great action localization performance by applying a recognition model followed by probability-based post-processing. Nevertheless, the probabilities provided by the recognition model frequently contain confused information causing challenge for post-processing. In this work, we adopt an action recognition model based on self-supervise learning to detect distracted activities and give potential action probabilities. Subsequently, a constraint ensemble strategy takes advantages of multi-camera views to provide robust predictions. Finally, we introduce a conditional post-processing operation to locate distracted behaviours and action temporal boundaries precisely. Experimenting on test set A2, our method obtains the sixth position on the public leaderboard of track 3 of the 2024 AI City Challenge.