Using Continual Learning for Real-Time Detection of Vulnerable Road Users in Complex Traffic Scenarios

TL;DR

This paper introduces a continual learning-enhanced YOLOv8-D system for real-time detection of vulnerable road users, improving accuracy and adaptability in complex traffic scenarios.

Contribution

It presents a novel integration of continual learning with YOLOv8 for adaptive VRU detection across diverse environments, addressing catastrophic forgetting.

Findings

YOLOv8x outperforms other models in F1 and mAP.

Continual learning improves detection accuracy across datasets.

Optimized gradient descent enhances model adaptability.

Abstract

Pedestrians and bicyclists are among the vulnerable road users (VRUs) that are inherently exposed to intricate traffic scenarios, which puts them at increased risk of sustaining injuries or facing fatal outcomes. This study presents an intelligent adaptive system that uses the YOLOv8-Dynamic (YOLOv8-D) algorithm that detects vulnerable road users and adapts in real time to prevent accidents before they occur. We select YOLOv8x as the detector by comparing it with other state-of-the-art object detection models, including Faster-RCNN, YOLOv5, YOLOv7, and variants. Compared to YOLOv5x, YOLOv8x shows improvements of 12.14% in F1 score and 45.61% in mean Average Precision (mAP). Against YOLOv7x, the improvements are 21.26% in F1 score and 128.44% in mAP. Our algorithm integrates continual learning ability in the architecture of the YOLOv8 detector to adjust to evolving road conditions flexibly, ensuring adaptability across multiple dataset domains and facilitating continuous enhancement of detection and tracking accuracy for VRUs, embracing the dynamic nature of real-world environments. In our proposed framework, we optimized the gradient descent mechanism of YOLOv8 model and train our optimized algorithm on two statistically different datasets in terms of image viewpoint and number of classes to achieve a 21.08% improvement in F1 score and a 31.86% improvement in mAP as compared to a custom YOLOv8 framework trained on a new dataset, thus overcoming the issue of catastrophic forgetting, which occurs when deep models are trained on statistically different types of datasets.