# Static Visual Spatial Priors for DoA Estimation

## TL;DR

This paper introduces a novel multi-modal DoA estimation method that leverages static visual spatial priors to improve sound source localization accuracy in complex environments.

## Contribution

It presents the first multi-modal DoA estimation approach using static visual priors to reduce false detections in acoustic scene analysis.

## Key findings

- Improved DoA accuracy over classic methods
- Effective suppression of false detections
- Validated on a new real-world dataset

## Abstract

As we interact with the world, for example when we communicate with our colleagues in a large open space or meeting room, we continuously analyse the surrounding environment and, in particular, localise and recognise acoustic events. While we largely take such abilities for granted, they represent a challenging problem for current robots or smart voice assistants as they can be easily fooled by high degree of sound interference in acoustically complex environments. Preventing such failures when using solely audio data is challenging, if not impossible since the algorithms need to take into account wider context and often understand the scene on a semantic level. In this paper, we propose what to our knowledge is the first multi-modal direction of arrival (DoA) of sound, which uses static visual spatial prior providing an auxiliary information about the environment to suppress some of the false DoA detections. We validate our approach on a newly collected real-world dataset, and show that our approach consistently improves over classic DoA baselines