Lightning NeRF: Efficient Hybrid Scene Representation for Autonomous Driving

TL;DR

Lightning NeRF introduces an efficient hybrid scene representation leveraging LiDAR data to significantly enhance view synthesis quality and speed in autonomous driving environments, addressing previous limitations in scene reconstruction and computational efficiency.

Contribution

It presents Lightning NeRF, a novel hybrid scene representation method that utilizes LiDAR priors to improve reconstruction quality and computational efficiency in outdoor autonomous driving scenes.

Findings

Outperforms state-of-the-art in novel view synthesis quality.

Achieves five-fold faster training speed.

Achieves ten-fold faster rendering speed.

Abstract

Recent studies have highlighted the promising application of NeRF in autonomous driving contexts. However, the complexity of outdoor environments, combined with the restricted viewpoints in driving scenarios, complicates the task of precisely reconstructing scene geometry. Such challenges often lead to diminished quality in reconstructions and extended durations for both training and rendering. To tackle these challenges, we present Lightning NeRF. It uses an efficient hybrid scene representation that effectively utilizes the geometry prior from LiDAR in autonomous driving scenarios. Lightning NeRF significantly improves the novel view synthesis performance of NeRF and reduces computational overheads. Through evaluations on real-world datasets, such as KITTI-360, Argoverse2, and our private dataset, we demonstrate that our approach not only exceeds the current state-of-the-art in novel view synthesis quality but also achieves a five-fold increase in training speed and a ten-fold improvement in rendering speed. Codes are available at https://github.com/VISION-SJTU/Lightning-NeRF .