Hybrid bundle-adjusting 3D Gaussians for view consistent rendering with pose optimization

TL;DR

This paper presents a hybrid 3D Gaussian model that jointly optimizes scene representation and camera poses, enabling view-consistent rendering even with pose inaccuracies, demonstrated through extensive experiments.

Contribution

The proposed model uniquely combines image-based and neural 3D representations for simultaneous rendering and pose optimization in view synthesis.

Findings

Effectively optimizes neural scene representations

Resolves significant camera pose misalignments

Demonstrates superior performance on real and synthetic datasets

Abstract

Novel view synthesis has made significant progress in the field of 3D computer vision. However, the rendering of view-consistent novel views from imperfect camera poses remains challenging. In this paper, we introduce a hybrid bundle-adjusting 3D Gaussians model that enables view-consistent rendering with pose optimization. This model jointly extract image-based and neural 3D representations to simultaneously generate view-consistent images and camera poses within forward-facing scenes. The effective of our model is demonstrated through extensive experiments conducted on both real and synthetic datasets. These experiments clearly illustrate that our model can effectively optimize neural scene representations while simultaneously resolving significant camera pose misalignments. The source code is available at https://github.com/Bistu3DV/hybridBA.