Vision-Only Gaussian Splatting for Collaborative Semantic Occupancy Prediction

TL;DR

This paper introduces a novel vision-only collaborative 3D semantic occupancy prediction method using sparse Gaussian splatting, which reduces communication costs and improves accuracy over existing dense or 2D-based methods.

Contribution

It is the first to leverage sparse 3D semantic Gaussian splatting for collaborative perception, enabling efficient, accurate, and communication-efficient 3D semantic occupancy prediction.

Findings

Outperforms single-agent perception by +8.42 mIoU

Outperforms baseline collaborative methods by +3.28 mIoU

Maintains robust performance with only 34.6% communication volume

Abstract

Collaborative perception enables connected vehicles to share information, overcoming occlusions and extending the limited sensing range inherent in single-agent (non-collaborative) systems. Existing vision-only methods for 3D semantic occupancy prediction commonly rely on dense 3D voxels, which incur high communication costs, or 2D planar features, which require accurate depth estimation or additional supervision, limiting their applicability to collaborative scenarios. To address these challenges, we propose the first approach leveraging sparse 3D semantic Gaussian splatting for collaborative 3D semantic occupancy prediction. By sharing and fusing intermediate Gaussian primitives, our method provides three benefits: a neighborhood-based cross-agent fusion that removes duplicates and suppresses noisy or inconsistent Gaussians; a joint encoding of geometry and semantics in each primitive, which reduces reliance on depth supervision and allows simple rigid alignment; and sparse, object-centric messages that preserve structural information while reducing communication volume. Extensive experiments demonstrate that our approach outperforms single-agent perception and baseline collaborative methods by +8.42 and +3.28 points in mIoU, and +5.11 and +22.41 points in IoU, respectively. When further reducing the number of transmitted Gaussians, our method still achieves a +1.9 improvement in mIoU, using only 34.6% communication volume, highlighting robust performance under limited communication budgets.