Real-Time Idling Vehicles Detection using Combined Audio-Visual Deep Learning

TL;DR

This paper introduces a real-time multi-sensor deep learning system that detects idling vehicles using audio-visual data, achieving high accuracy in identifying engine status changes to help reduce vehicle pollution.

Contribution

It presents a novel real-time detection algorithm combining visual motion analysis and contrastive-learning-based audio classification for vehicle idling detection.

Findings

Achieves 71.02% average precision for idle detection.

Achieves 91.06% precision for engine off detection.

Successfully tested in a real-world hospital drop-off scenario.

Abstract

Combustion vehicle emissions contribute to poor air quality and release greenhouse gases into the atmosphere, and vehicle pollution has been associated with numerous adverse health effects. Roadways with extensive waiting and/or passenger drop off, such as schools and hospital drop-off zones, can result in high incidence and density of idling vehicles. This can produce micro-climates of increased vehicle pollution. Thus, the detection of idling vehicles can be helpful in monitoring and responding to unnecessary idling and be integrated into real-time or off-line systems to address the resulting pollution. In this paper we present a real-time, dynamic vehicle idling detection algorithm. The proposed idle detection algorithm and notification rely on an algorithm to detect these idling vehicles. The proposed method relies on a multi-sensor, audio-visual, machine-learning workflow to detect idling vehicles visually under three conditions: moving, static with the engine on, and static with the engine off. The visual vehicle motion detector is built in the first stage, and then a contrastive-learning-based latent space is trained for classifying static vehicle engine sound. We test our system in real-time at a hospital drop-off point in Salt Lake City. This in-situ dataset was collected and annotated, and it includes vehicles of varying models and types. The experiments show that the method can detect engine switching on or off instantly and achieves 71.02 average precision (AP) for idle detections and 91.06 for engine off detections.