Improving the Environmental Perception of Autonomous Vehicles using Deep Learning-based Audio Classification

TL;DR

This paper introduces a deep learning-based audio classification framework that enhances autonomous vehicles' environmental perception by accurately identifying relevant sounds like sirens and horns, complementing visual sensors.

Contribution

It presents a novel CNN-based audio classification system trained on UrbanSound8k, achieving high accuracy and outperforming existing frameworks for AV-related sounds.

Findings

Achieved 97.82% accuracy in classifying seven relevant audio classes.

Demonstrated improved performance over existing audio classification methods.

Validated the framework's effectiveness in urban environment scenarios.

Abstract

Sense of hearing is crucial for autonomous vehicles (AVs) to better perceive its surrounding environment. Although visual sensors of an AV, such as camera, lidar, and radar, help to see its surrounding environment, an AV cannot see beyond those sensors line of sight. On the other hand, an AV s sense of hearing cannot be obstructed by line of sight. For example, an AV can identify an emergency vehicle s siren through audio classification even though the emergency vehicle is not within the line of sight of the AV. Thus, auditory perception is complementary to the camera, lidar, and radar-based perception systems. This paper presents a deep learning-based robust audio classification framework aiming to achieve improved environmental perception for AVs. The presented framework leverages a deep Convolution Neural Network (CNN) to classify different audio classes. UrbanSound8k, an urban environment dataset, is used to train and test the developed framework. Seven audio classes i.e., air conditioner, car horn, children playing, dog bark, engine idling, gunshot, and siren, are identified from the UrbanSound8k dataset because of their relevancy related to AVs. Our framework can classify different audio classes with 97.82% accuracy. Moreover, the audio classification accuracies with all ten classes are presented, which proves that our framework performed better in the case of AV-related sounds compared to the existing audio classification frameworks.