GFreeDet: Exploiting Gaussian Splatting and Foundation Models for Model-free Unseen Object Detection in the BOP Challenge 2024

TL;DR

GFreeDet is a novel model-free unseen object detection method that uses Gaussian splatting and foundation models to reconstruct objects from videos, achieving competitive results in the BOP Challenge 2024.

Contribution

It introduces a model-free approach for unseen object detection that reconstructs objects directly from videos without relying on CAD templates.

Findings

Achieved competitive performance on BOP-H3 benchmark.

Won best overall and best fast method awards at BOP Challenge 2024.

Demonstrated robustness in detecting novel objects in mixed reality applications.

Abstract

We present GFreeDet, an unseen object detection approach that leverages Gaussian splatting and vision Foundation models under model-free setting. Unlike existing methods that rely on predefined CAD templates, GFreeDet reconstructs objects directly from reference videos using Gaussian splatting, enabling robust detection of novel objects without prior 3D models. Evaluated on the BOP-H3 benchmark, GFreeDet achieves comparable performance to CAD-based methods, demonstrating the viability of model-free detection for mixed reality (MR) applications. Notably, GFreeDet won the best overall method and the best fast method awards in the model-free 2D detection track at BOP Challenge 2024.