MVP: Robust Multi-View Practice for Driving Action Localization

TL;DR

This paper introduces MVP, a robust multi-view approach for localizing driving actions in videos, leveraging synchronized multi-view data, fine-tuned feature extractors, and elaborate post-processing to improve accuracy in a challenging dataset.

Contribution

The paper proposes a novel multi-view practice framework that effectively localizes driving actions using synchronized videos, fine-tuned feature extractors, and advanced post-processing techniques.

Findings

Achieves 28.49% F1-score on the Track3 test set.

Demonstrates robustness in localizing diverse driving actions.

Utilizes multi-view synchronization and tailored post-processing for improved accuracy.

Abstract

Distracted driving causes thousands of deaths per year, and how to apply deep-learning methods to prevent these tragedies has become a crucial problem. In Track3 of the 6th AI City Challenge, researchers provide a high-quality video dataset with densely action annotations. Due to the small data scale and unclear action boundary, the dataset presents a unique challenge to precisely localize all the different actions and classify their categories. In this paper, we make good use of the multi-view synchronization among videos, and conduct robust Multi-View Practice (MVP) for driving action localization. To avoid overfitting, we fine-tune SlowFast with Kinetics-700 pre-training as the feature extractor. Then the features of different views are passed to ActionFormer to generate candidate action proposals. For precisely localizing all the actions, we design elaborate post-processing, including model voting, threshold filtering and duplication removal. The results show that our MVP is robust for driving action localization, which achieves 28.49% F1-score in the Track3 test set.