3D Scene-Camera Representation with Joint Camera Photometric Optimization

TL;DR

This paper introduces a joint camera photometric optimization method for 3D scene representation that effectively accounts for photometric distortions, improving quality under degraded imaging conditions.

Contribution

It proposes a novel full photometric model and camera representation that separates scene-unrelated information, enhancing 3D scene accuracy and robustness.

Findings

Achieves high-quality 3D scene representations under imaging degradation.

Effectively separates scene-unrelated information from the scene.

Improves robustness against vignetting and dirt effects.

Abstract

Representing scenes from multi-view images is a crucial task in computer vision with extensive applications. However, inherent photometric distortions in the camera imaging can significantly degrade image quality. Without accounting for these distortions, the 3D scene representation may inadvertently incorporate erroneous information unrelated to the scene, diminishing the quality of the representation. In this paper, we propose a novel 3D scene-camera representation with joint camera photometric optimization. By introducing internal and external photometric model, we propose a full photometric model and corresponding camera representation. Based on simultaneously optimizing the parameters of the camera representation, the proposed method effectively separates scene-unrelated information from the 3D scene representation. Additionally, during the optimization of the photometric parameters, we introduce a depth regularization to prevent the 3D scene representation from fitting scene-unrelated information. By incorporating the camera model as part of the mapping process, the proposed method constructs a complete map that includes both the scene radiance field and the camera photometric model. Experimental results demonstrate that the proposed method can achieve high-quality 3D scene representations, even under conditions of imaging degradation, such as vignetting and dirt.