UniDet3D: Multi-dataset Indoor 3D Object Detection

TL;DR

UniDet3D introduces a multi-dataset training approach for indoor 3D object detection, leveraging a transformer-based architecture to improve performance across diverse indoor environments and benchmarks.

Contribution

The paper presents UniDet3D, a novel multi-dataset training framework with a transformer-based model that unifies label spaces for enhanced indoor 3D object detection.

Findings

Significant performance improvements on 6 indoor benchmarks.

Effective multi-dataset training with unified label spaces.

Transformer-based architecture facilitates practical deployment.

Abstract

Growing customer demand for smart solutions in robotics and augmented reality has attracted considerable attention to 3D object detection from point clouds. Yet, existing indoor datasets taken individually are too small and insufficiently diverse to train a powerful and general 3D object detection model. In the meantime, more general approaches utilizing foundation models are still inferior in quality to those based on supervised training for a specific task. In this work, we propose \ours{}, a simple yet effective 3D object detection model, which is trained on a mixture of indoor datasets and is capable of working in various indoor environments. By unifying different label spaces, \ours{} enables learning a strong representation across multiple datasets through a supervised joint training scheme. The proposed network architecture is built upon a vanilla transformer encoder, making it easy to run, customize and extend the prediction pipeline for practical use. Extensive experiments demonstrate that \ours{} obtains significant gains over existing 3D object detection methods in 6 indoor benchmarks: ScanNet (+1.1 mAP50), ARKitScenes (+19.4 mAP25), S3DIS (+9.1 mAP50), MultiScan (+9.3 mAP50), 3RScan (+3.2 mAP50), and ScanNet++ (+2.7 mAP50). Code is available at https://github.com/filapro/unidet3d .