Unifying Appearance Codes and Bilateral Grids for Driving Scene Gaussian Splatting

TL;DR

This paper introduces a multi-scale bilateral grid that combines appearance codes and bilateral grids to enhance geometric accuracy in neural rendering of driving scenes, outperforming previous methods across multiple datasets.

Contribution

The paper presents a novel multi-scale bilateral grid approach that unifies appearance codes and bilateral grids, significantly improving scene reconstruction accuracy in autonomous driving scenarios.

Findings

Outperforms previous methods on Waymo, NuScenes, Argoverse, and PandaSet datasets.

Reduces floaters caused by photometric inconsistency.

Enhances geometric accuracy in dynamic driving scenes.

Abstract

Neural rendering techniques, including NeRF and Gaussian Splatting (GS), rely on photometric consistency to produce high-quality reconstructions. However, in real-world scenarios, it is challenging to guarantee perfect photometric consistency in acquired images. Appearance codes have been widely used to address this issue, but their modeling capability is limited, as a single code is applied to the entire image. Recently, the bilateral grid was introduced to perform pixel-wise color mapping, but it is difficult to optimize and constrain effectively. In this paper, we propose a novel multi-scale bilateral grid that unifies appearance codes and bilateral grids. We demonstrate that this approach significantly improves geometric accuracy in dynamic, decoupled autonomous driving scene reconstruction, outperforming both appearance codes and bilateral grids. This is crucial for autonomous driving, where accurate geometry is important for obstacle avoidance and control. Our method shows strong results across four datasets: Waymo, NuScenes, Argoverse, and PandaSet. We further demonstrate that the improvement in geometry is driven by the multi-scale bilateral grid, which effectively reduces floaters caused by photometric inconsistency.