Sensor array and camera fusion via unbalanced optimal transport for 3D source localization

TL;DR

This paper introduces a novel sensor-camera fusion method using unbalanced optimal transport for accurate 3D source localization, emphasizing efficiency and modularity without training data.

Contribution

It extends covariance matrix fitting with unbalanced optimal transport, enabling flexible sensor-data alignment and efficient large-scale 3D source localization.

Findings

Improved localization accuracy over sensor-only methods.

Efficient greedy coordinate descent algorithm for large-scale problems.

Validated on real acoustic array data.

Abstract

We address the problem of localizing multiple sources in 3D by combining sensor array measurements with camera observations. We propose a fusion framework extending the covariance matrix fitting method with an unbalanced optimal transport regularization term that softly aligns sensor array responses with visual priors while allowing flexibility in mass allocation. To solve the resulting largescale problem, we adopt a greedy coordinate descent algorithm that efficiently updates the transport plan. Its computational efficiency makes full 3D localization feasible in practice. The proposed framework is modular and does not rely on labeled data or training, in contrast with deep learning-based fusion approaches. Although validated here on acoustic arrays, the method is general to arbitrary sensor arrays. Experiments on real data show that the proposed approach improves localization accuracy compared to sensor-only baselines.