A Synthetic Benchmark for Collaborative 3D Semantic Occupancy Prediction in V2X-Enabled Autonomous Driving

TL;DR

This paper introduces a synthetic benchmark dataset and a baseline model for collaborative 3D semantic occupancy prediction in autonomous driving, demonstrating improved accuracy through inter-agent feature fusion and extended prediction ranges.

Contribution

It provides the first high-resolution synthetic dataset and a baseline model for collaborative 3D semantic occupancy prediction in V2X-enabled autonomous driving.

Findings

Baseline model outperforms existing methods.

Performance improves with increased prediction range.

Benchmark facilitates systematic evaluation of collaborative perception.

Abstract

3D semantic occupancy prediction is an emerging perception paradigm in autonomous driving, providing a voxel-level representation of both geometric details and semantic categories. However, its effectiveness is inherently constrained in single-vehicle setups by occlusions, restricted sensor range, and narrow viewpoints. To address these limitations, collaborative perception enables the exchange of complementary information, thereby enhancing the completeness and accuracy of predictions. Despite its potential, research on collaborative 3D semantic occupancy prediction is hindered by the lack of dedicated datasets. To bridge this gap, we design a high-resolution semantic voxel sensor in CARLA to produce dense and comprehensive annotations. We further develop a baseline model that performs inter-agent feature fusion via spatial alignment and attention aggregation. In addition, we establish benchmarks with varying prediction ranges designed to systematically assess the impact of spatial extent on collaborative prediction. Experimental results demonstrate the superior performance of our baseline, with increasing gains observed as range expands. Our code is available at https://github.com/tlab-wide/Co3SOP}{https://github.com/tlab-wide/Co3SOP.