LuSh-NeRF: Lighting up and Sharpening NeRFs for Low-light Scenes

TL;DR

LuSh-NeRF is a novel method that reconstructs sharp, noise-free NeRFs from low-light, hand-held images by modeling noise and blur sequentially, leveraging multi-view consistency and frequency analysis.

Contribution

The paper introduces LuSh-NeRF, which uniquely models noise and blur sequentially for low-light NeRF reconstruction, including novel modules for noise decoupling and camera motion prediction.

Findings

Outperforms existing low-light NeRF methods.

Effectively decouples noise from scene representation.

Accurately predicts camera trajectories in low-light conditions.

Abstract

Neural Radiance Fields (NeRFs) have shown remarkable performances in producing novel-view images from high-quality scene images. However, hand-held low-light photography challenges NeRFs as the captured images may simultaneously suffer from low visibility, noise, and camera shakes. While existing NeRF methods may handle either low light or motion, directly combining them or incorporating additional image-based enhancement methods does not work as these degradation factors are highly coupled. We observe that noise in low-light images is always sharp regardless of camera shakes, which implies an implicit order of these degradation factors within the image formation process. To this end, we propose in this paper a novel model, named LuSh-NeRF, which can reconstruct a clean and sharp NeRF from a group of hand-held low-light images. The key idea of LuSh-NeRF is to sequentially model noise and blur in the images via multi-view feature consistency and frequency information of NeRF, respectively. Specifically, LuSh-NeRF includes a novel Scene-Noise Decomposition (SND) module for decoupling the noise from the scene representation and a novel Camera Trajectory Prediction (CTP) module for the estimation of camera motions based on low-frequency scene information. To facilitate training and evaluations, we construct a new dataset containing both synthetic and real images. Experiments show that LuSh-NeRF outperforms existing approaches. Our code and dataset can be found here: https://github.com/quzefan/LuSh-NeRF.