CatFree3D: Category-agnostic 3D Object Detection with Diffusion

TL;DR

CatFree3D presents a diffusion-based, category-agnostic 3D object detection pipeline that improves accuracy and generalisation, introducing a new evaluation metric for better assessment of detection results.

Contribution

The paper introduces a novel diffusion-based pipeline for category-agnostic 3D detection and the Normalised Hungarian Distance metric for improved evaluation.

Findings

Achieves state-of-the-art accuracy in 3D detection

Demonstrates strong generalisation across datasets

Outperforms traditional IoU-based metrics

Abstract

Image-based 3D object detection is widely employed in applications such as autonomous vehicles and robotics, yet current systems struggle with generalisation due to complex problem setup and limited training data. We introduce a novel pipeline that decouples 3D detection from 2D detection and depth prediction, using a diffusion-based approach to improve accuracy and support category-agnostic detection. Additionally, we introduce the Normalised Hungarian Distance (NHD) metric for an accurate evaluation of 3D detection results, addressing the limitations of traditional IoU and GIoU metrics. Experimental results demonstrate that our method achieves state-of-the-art accuracy and strong generalisation across various object categories and datasets.