FlexDrive: Toward Trajectory Flexibility in Driving Scene Reconstruction and Rendering

TL;DR

FlexDrive introduces an inverse view warping technique with a depth bootstrap strategy to improve out-of-path scene reconstruction and rendering, achieving superior results on the Waymo dataset.

Contribution

The paper presents a novel inverse view warping method with a depth bootstrap approach to enhance out-of-path scene reconstruction in driving scenarios.

Findings

Outperforms previous methods on Waymo dataset

Achieves high-quality in-path and out-of-path rendering

Introduces a new benchmark for out-of-path rendering evaluation

Abstract

Driving scene reconstruction and rendering have advanced significantly using the 3D Gaussian Splatting. However, most prior research has focused on the rendering quality along a pre-recorded vehicle path and struggles to generalize to out-of-path viewpoints, which is caused by the lack of high-quality supervision in those out-of-path views. To address this issue, we introduce an Inverse View Warping technique to create compact and high-quality images as supervision for the reconstruction of the out-of-path views, enabling high-quality rendering results for those views. For accurate and robust inverse view warping, a depth bootstrap strategy is proposed to obtain on-the-fly dense depth maps during the optimization process, overcoming the sparsity and incompleteness of LiDAR depth data. Our method achieves superior in-path and out-of-path reconstruction and rendering performance on the widely used Waymo Open dataset. In addition, a simulator-based benchmark is proposed to obtain the out-of-path ground truth and quantitatively evaluate the performance of out-of-path rendering, where our method outperforms previous methods by a significant margin.