Scaling Point-based Differentiable Rendering for Large-scale Reconstruction

TL;DR

This paper introduces Gaian, a versatile distributed training system for Point-based Differentiable Rendering that significantly reduces communication overhead and enhances training efficiency for large-scale 3D scene reconstruction.

Contribution

Gaian provides a unified API for PBDR methods and optimizes data locality, enabling scalable and efficient distributed training across multiple algorithms and datasets.

Findings

Reduces communication by up to 91%

Improves training throughput by 1.50x-3.71x

Supports multiple PBDR algorithms efficiently

Abstract

Point-based Differentiable Rendering (PBDR) enables high-fidelity 3D scene reconstruction, but scaling PBDR to high-resolution and large scenes requires efficient distributed training systems. Existing systems are tightly coupled to a specific PBDR method. And they suffer from severe communication overhead due to poor data locality. In this paper, we present Gaian, a general distributed training system for PBDR. Gaian provides a unified API expressive enough to support existing PBDR methods, while exposing rich data-access information, which Gaian leverages to optimize locality and reduce communication. We evaluated Gaian by implementing 4 PBDR algorithms. Our implementations achieve high performance and resource efficiency: across six datasets and up to 128 GPUs, it reduces communication by up to 91% and improves training throughput by 1.50x-3.71x.