Object Gaussian for Monocular 6D Pose Estimation from Sparse Views

TL;DR

SGPose is a novel framework that estimates 6D object poses from as few as ten sparse views without relying on CAD models, using Gaussian-based representations and geometric supervision.

Contribution

It introduces a CAD-model-free, Gaussian-based approach for sparse view 6D pose estimation that outperforms existing methods on benchmark datasets.

Findings

Outperforms existing methods on Occlusion LM-O dataset

Effective with as few as ten input views

Does not rely on CAD models or SfM pipeline geometry

Abstract

Monocular object pose estimation, as a pivotal task in computer vision and robotics, heavily depends on accurate 2D-3D correspondences, which often demand costly CAD models that may not be readily available. Object 3D reconstruction methods offer an alternative, among which recent advancements in 3D Gaussian Splatting (3DGS) afford a compelling potential. Yet its performance still suffers and tends to overfit with fewer input views. Embracing this challenge, we introduce SGPose, a novel framework for sparse view object pose estimation using Gaussian-based methods. Given as few as ten views, SGPose generates a geometric-aware representation by starting with a random cuboid initialization, eschewing reliance on Structure-from-Motion (SfM) pipeline-derived geometry as required by traditional 3DGS methods. SGPose removes the dependence on CAD models by regressing dense 2D-3D correspondences between images and the reconstructed model from sparse input and random initialization, while the geometric-consistent depth supervision and online synthetic view warping are key to the success. Experiments on typical benchmarks, especially on the Occlusion LM-O dataset, demonstrate that SGPose outperforms existing methods even under sparse view constraints, under-scoring its potential in real-world applications.