GTR: Gaussian Splatting Tracking and Reconstruction of Unknown Objects Based on Appearance and Geometric Complexity

TL;DR

This paper introduces a novel adaptive method combining Gaussian Splatting and hybrid tracking to improve 6-DoF object tracking and 3D reconstruction from monocular RGBD videos, especially for complex and symmetric objects.

Contribution

The paper presents a new approach that integrates 3D Gaussian Splatting with hybrid geometry and appearance tracking, along with a benchmark for challenging object classes.

Findings

Achieves robust tracking across diverse objects.

Produces high-fidelity 3D reconstructions.

Sets new standards for single-sensor 3D reconstruction.

Abstract

We present a novel method for 6-DoF object tracking and high-quality 3D reconstruction from monocular RGBD video. Existing methods, while achieving impressive results, often struggle with complex objects, particularly those exhibiting symmetry, intricate geometry or complex appearance. To bridge these gaps, we introduce an adaptive method that combines 3D Gaussian Splatting, hybrid geometry/appearance tracking, and key frame selection to achieve robust tracking and accurate reconstructions across a diverse range of objects. Additionally, we present a benchmark covering these challenging object classes, providing high-quality annotations for evaluating both tracking and reconstruction performance. Our approach demonstrates strong capabilities in recovering high-fidelity object meshes, setting a new standard for single-sensor 3D reconstruction in open-world environments.