SPARK: Scalable Real-Time Point Cloud Aggregation with Multi-View Self-Calibration

TL;DR

SPARK is a real-time, scalable multi-camera 3D reconstruction framework that self-calibrates extrinsics and fuses point clouds effectively, enabling stable and accurate 3D perception in dynamic scenes.

Contribution

It introduces a novel self-calibrating framework combining online extrinsic estimation and confidence-driven point cloud fusion for large-scale multi-camera systems.

Findings

Outperforms existing methods in extrinsic accuracy.

Achieves stable point clouds in dynamic scenes.

Operates in real-time with linear scalability.

Abstract

Real-time multi-camera 3D reconstruction is crucial for 3D perception, immersive interaction, and robotics. Existing methods struggle with multi-view fusion, camera extrinsic uncertainty, and scalability for large camera setups. We propose SPARK, a self-calibrating real-time multi-camera point cloud reconstruction framework that jointly handles point cloud fusion and extrinsic uncertainty. SPARK consists of: (1) a geometry-aware online extrinsic estimation module leveraging multi-view priors and enforcing cross-view and temporal consistency for stable self-calibration, and (2) a confidence-driven point cloud fusion strategy modeling depth reliability and visibility at pixel and point levels to suppress noise and view-dependent inconsistencies. By performing frame-wise fusion without accumulation, SPARK produces stable point clouds in dynamic scenes while scaling linearly with the number of cameras. Extensive experiments on real-world multi-camera systems show that SPARK outperforms existing approaches in extrinsic accuracy, geometric consistency, temporal stability, and real-time performance, demonstrating its effectiveness and scalability for large-scale multi-camera 3D reconstruction.