Look, Radiate, and Learn: Self-Supervised Localisation via Radio-Visual Correspondence

TL;DR

This paper introduces MaxRay, a synthetic dataset and benchmark for radio-visual target localization, and demonstrates self-supervised learning of radio target localization from paired data without labels, advancing radio sensing capabilities.

Contribution

The paper presents MaxRay, a novel synthetic dataset and benchmark, and proposes a self-supervised method for radio target localization using radio-visual correspondence.

Findings

Self-supervised radio localization achieves competitive accuracy.

Paired radio-visual data enables label-free training.

MaxRay dataset facilitates systematic evaluation of radio sensing methods.

Abstract

Next generation cellular networks will implement radio sensing functions alongside customary communications, thereby enabling unprecedented worldwide sensing coverage outdoors. Deep learning has revolutionised computer vision but has had limited application to radio perception tasks, in part due to lack of systematic datasets and benchmarks dedicated to the study of the performance and promise of radio sensing. To address this gap, we present MaxRay: a synthetic radio-visual dataset and benchmark that facilitate precise target localisation in radio. We further propose to learn to localise targets in radio without supervision by extracting self-coordinates from radio-visual correspondence. We use such self-supervised coordinates to train a radio localiser network. We characterise our performance against a number of state-of-the-art baselines. Our results indicate that accurate radio target localisation can be automatically learned from paired radio-visual data without labels, which is important for empirical data. This opens the door for vast data scalability and may prove key to realising the promise of robust radio sensing atop a unified communication-perception cellular infrastructure. Dataset will be hosted on IEEE DataPort.