SynthSoM: A synthetic intelligent multi-modal sensing-communication dataset for Synesthesia of Machines (SoM)

TL;DR

SynthSoM introduces a comprehensive synthetic multi-modal sensing-communication dataset for machine synesthesia research, integrating high-precision simulation platforms to enable diverse, multi-condition scenarios for advancing sensing-communication integration.

Contribution

A novel simulation platform combining AirSim, WaveFarer, and Wireless InSite to generate SynthSoM, a multi-modal dataset for sensing-communication research with diverse scenarios and data types.

Findings

Dataset validated with statistical and ML evaluation metrics.

SynthSoM covers diverse weather, time, and agent density conditions.

Open-sourced for benchmarking and algorithm development.

Abstract

Given the importance of datasets for sensing-communication integration research, a novel simulation platform for constructing communication and multi-modal sensory dataset is developed. The developed platform integrates three high-precision software, i.e., AirSim, WaveFarer, and Wireless InSite, and further achieves in-depth integration and precise alignment of them. Based on the developed platform, a new synthetic intelligent multi-modal sensing-communication dataset for Synesthesia of Machines (SoM), named SynthSoM, is proposed. The SynthSoM dataset contains various air-ground multi-link cooperative scenarios with comprehensive conditions, including multiple weather conditions, times of the day, intelligent agent densities, frequency bands, and antenna types. The SynthSoM dataset encompasses multiple data modalities, including radio-frequency (RF) channel large-scale and small-scale fading data, RF millimeter wave (mmWave) radar sensory data, and non-RF sensory data, e.g., RGB images, depth maps, and light detection and ranging (LiDAR) point clouds. The quality of SynthSoM dataset is validated via statistics-based qualitative inspection and evaluation metrics through machine learning (ML) via real-world measurements. The SynthSoM dataset is open-sourced and provides consistent data for cross-comparing SoM-related algorithms.