# An auditory-visual cooperative perception method for honking vehicle localization

## TL;DR

This paper introduces a method that combines audio and visual data to more accurately locate honking vehicles, improving over traditional sound-based approaches.

## Contribution

The novel AVCP method integrates ECAPA-TDNN and YOLO v9 for cooperative auditory-visual honking vehicle localization.

## Key findings

- The AVCP method is less affected by environmental noise compared to traditional methods.
- It achieves more accurate and distant vehicle identification and localization.
- Experiments on simulated and public datasets validated the method's effectiveness.

## Abstract

Locating honking vehicles is crucial for controlling arbitrary honking and reducing environmental noise. However, traditional methods for honking vehicle localization, which utilize sound source localization technology, suffer from inaccuracies and limited detection range due to the multipath effects of sound propagation and environmental noise interference. To address these challenges, an auditory-visual cooperative perception (AVCP) method for honking vehicle localization is proposed, and a detailed workflow of this method is presented. In the AVCP method workflow, the Emphasized Channel Attention, Propagation, and Aggregation in Time-Delay Neural Network (ECAPA-TDNN) is used to recognize honking vehicle models from captured audio signals, as different vehicle models exhibit distinct horn sound characteristics. Subsequently, YOLO v9 is employed to detect vehicles and recognize their corresponding models in the images captured by the camera. Thus, among the vehicles detected and identified using YOLO v9, the honking vehicle is determined as the one whose model matches the vehicle model recognized by ECAPA-TDNN. Additionally, experiments with simulated and public datasets were conducted to evaluate the performance of the AVCP method for honking vehicle localization. The experimental results show that the AVCP method is less susceptible to environmental noise and can more accurately identify and locate vehicles from greater distances compared to traditional methods based on sound source localization technology.