Veta-GS: View-dependent deformable 3D Gaussian Splatting for thermal infrared Novel-view Synthesis

TL;DR

Veta-GS introduces a view-dependent deformable 3D Gaussian Splatting method for thermal infrared novel-view synthesis, addressing transmission effects, emissivity, and low resolution issues to improve rendering quality.

Contribution

It proposes a novel view-dependent deformation field and Thermal Feature Extractor to enhance thermal IR view synthesis robustness and accuracy.

Findings

Outperforms existing methods on TI-NSD benchmark

Effectively captures subtle thermal variations

Reduces floaters and blur in rendered images

Abstract

Recently, 3D Gaussian Splatting (3D-GS) based on Thermal Infrared (TIR) imaging has gained attention in novel-view synthesis, showing real-time rendering. However, novel-view synthesis with thermal infrared images suffers from transmission effects, emissivity, and low resolution, leading to floaters and blur effects in rendered images. To address these problems, we introduce Veta-GS, which leverages a view-dependent deformation field and a Thermal Feature Extractor (TFE) to precisely capture subtle thermal variations and maintain robustness. Specifically, we design view-dependent deformation field that leverages camera position and viewing direction, which capture thermal variations. Furthermore, we introduce the Thermal Feature Extractor (TFE) and MonoSSIM loss, which consider appearance, edge, and frequency to maintain robustness. Extensive experiments on the TI-NSD benchmark show that our method achieves better performance over existing methods.