SimART: A Unified and Open Real-world Multimodal Simulation Platform for 6G Integrated Sensing and Communication

TL;DR

SimART is an integrated, reproducible simulation platform combining robotics, ray tracing, and wireless tools for 6G sensing and communication research, supporting diverse environments and multimodal data streams.

Contribution

It introduces a unified ROS-based pipeline that synchronizes multimodal streams and enables plug-and-play simulation components for 6G ISAC research.

Findings

Reproducible simulation pipeline with synchronized multimodal streams.

Scene construction converts maps into visual and electromagnetic assets.

Case study demonstrates improved beam prediction using the platform.

Abstract

Research on sixth-generation (6G) integrated sensing and communication (ISAC) increasingly depends on multimodal datasets. These datasets need to jointly characterize wireless propagation, onboard sensing, and platform mobility. Existing tools cover only part of these aspects. Robotics simulators model physics and perception but not site-specific channels, while ray tracing and link level tools lack vehicle dynamics and onboard sensors. Combining them manually leads to workflows that are fragile and hard to reproduce. Rather than introducing another standalone simulator, this article presents SimART. It integrates mature robotics, ray tracing, and wireless evaluation engines into a single reproducible pipeline. The key idea is a robot operating system (ROS) backbone that both synchronizes and organizes all multimodal streams. A shared clock, a common coordinate frame, and timestamped messages keep the streams aligned in time and space, and a single rosbag recording captures the full session into one reproducible file. This design decouples the sensing front end from the wireless back end, so that any ROS-compatible simulator can be plugged in while reusing the same back end across aerial, ground, indoor, and maritime ISAC settings. On top of this backbone, SimART contributes a scene construction pipeline that converts both OpenStreetMap extracts and user-defined layouts into spatially aligned visual and electromagnetic assets, and a channel knowledge map (CKM) generator that aggregates ray tracing and system level outputs into spatial priors for ISAC algorithms. A case study on vision and position aided beam prediction demonstrates the utility of the platform. The code is publicly available at https://github.com/guchuanv-alt/SimART.