HazardNet: A Small-Scale Vision Language Model for Real-Time Traffic Safety Detection at Edge Devices

TL;DR

HazardNet is a compact vision-language model fine-tuned for real-time traffic safety detection on edge devices, utilizing a new VQA dataset to outperform larger models in accuracy and efficiency.

Contribution

This work introduces HazardNet, a small-scale vision-language model optimized for edge deployment, and HazardQA, a specialized dataset for safety-critical traffic scenarios.

Findings

HazardNet achieved up to 89% improvement in F1-Score over the base model.

HazardNet's performance is comparable to larger models like GPT-4o, with some cases up to 6% better.

The model enables real-time traffic safety detection on resource-constrained edge devices.

Abstract

Traffic safety remains a vital concern in contemporary urban settings, intensified by the increase of vehicles and the complicated nature of road networks. Traditional safety-critical event detection systems predominantly rely on sensor-based approaches and conventional machine learning algorithms, necessitating extensive data collection and complex training processes to adhere to traffic safety regulations. This paper introduces HazardNet, a small-scale Vision Language Model designed to enhance traffic safety by leveraging the reasoning capabilities of advanced language and vision models. We built HazardNet by fine-tuning the pre-trained Qwen2-VL-2B model, chosen for its superior performance among open-source alternatives and its compact size of two billion parameters. This helps to facilitate deployment on edge devices with efficient inference throughput. In addition, we present HazardQA, a novel Vision Question Answering (VQA) dataset constructed specifically for training HazardNet on real-world scenarios involving safety-critical events. Our experimental results show that the fine-tuned HazardNet outperformed the base model up to an 89% improvement in F1-Score and has comparable results with improvement in some cases reach up to 6% when compared to larger models, such as GPT-4o. These advancements underscore the potential of HazardNet in providing real-time, reliable traffic safety event detection, thereby contributing to reduced accidents and improved traffic management in urban environments. Both HazardNet model and the HazardQA dataset are available at https://huggingface.co/Tami3/HazardNet and https://huggingface.co/datasets/Tami3/HazardQA, respectively.