FreeDriveRF: Monocular RGB Dynamic NeRF without Poses for Autonomous Driving via Point-Level Dynamic-Static Decoupling

TL;DR

FreeDriveRF enables dynamic scene reconstruction for autonomous driving using only monocular RGB images without pose data, leveraging semantic supervision and optical flow to improve accuracy and reduce system complexity.

Contribution

It introduces a pose-free dynamic NeRF method that decouples static and dynamic scene parts using semantic cues and optical flow, enhancing robustness in monocular autonomous driving scenarios.

Findings

Outperforms existing methods on KITTI and Waymo datasets

Achieves accurate dynamic scene reconstruction without pose inputs

Reduces system complexity by eliminating multi-sensor requirements

Abstract

Dynamic scene reconstruction for autonomous driving enables vehicles to perceive and interpret complex scene changes more precisely. Dynamic Neural Radiance Fields (NeRFs) have recently shown promising capability in scene modeling. However, many existing methods rely heavily on accurate poses inputs and multi-sensor data, leading to increased system complexity. To address this, we propose FreeDriveRF, which reconstructs dynamic driving scenes using only sequential RGB images without requiring poses inputs. We innovatively decouple dynamic and static parts at the early sampling level using semantic supervision, mitigating image blurring and artifacts. To overcome the challenges posed by object motion and occlusion in monocular camera, we introduce a warped ray-guided dynamic object rendering consistency loss, utilizing optical flow to better constrain the dynamic modeling process. Additionally, we incorporate estimated dynamic flow to constrain the pose optimization process, improving the stability and accuracy of unbounded scene reconstruction. Extensive experiments conducted on the KITTI and Waymo datasets demonstrate the superior performance of our method in dynamic scene modeling for autonomous driving.