Evaluating Vision-Language Models for Zero-Shot Detection, Classification, and Association of Motorcycles, Passengers, and Helmets

TL;DR

This paper assesses the effectiveness of the OWLv2 vision-language model in zero-shot detection and classification of motorcycle occupants and helmet usage from video data, aiming to improve traffic safety monitoring.

Contribution

It introduces a cascaded detection and classification approach using OWLv2 and CNNs, demonstrating zero-shot learning capabilities in a traffic safety context with extended datasets.

Findings

Average precision of 0.5324 for helmet detection

Effective zero-shot detection under varied conditions

Potential for improved traffic safety enforcement

Abstract

Motorcycle accidents pose significant risks, particularly when riders and passengers do not wear helmets. This study evaluates the efficacy of an advanced vision-language foundation model, OWLv2, in detecting and classifying various helmet-wearing statuses of motorcycle occupants using video data. We extend the dataset provided by the CVPR AI City Challenge and employ a cascaded model approach for detection and classification tasks, integrating OWLv2 and CNN models. The results highlight the potential of zero-shot learning to address challenges arising from incomplete and biased training datasets, demonstrating the usage of such models in detecting motorcycles, helmet usage, and occupant positions under varied conditions. We have achieved an average precision of 0.5324 for helmet detection and provided precision-recall curves detailing the detection and classification performance. Despite limitations such as low-resolution data and poor visibility, our research shows promising advancements in automated vehicle safety and traffic safety enforcement systems.