MVSDet: Multi-View Indoor 3D Object Detection via Efficient Plane Sweeps

TL;DR

MVSDet introduces a plane sweep-based approach for indoor 3D object detection that improves accuracy by effectively utilizing probabilistic sampling and Gaussian Splatting, outperforming NeRF-based methods.

Contribution

The paper proposes a novel plane sweep method with probabilistic sampling and Gaussian Splatting for more accurate geometry-aware 3D detection in indoor scenes.

Findings

Outperforms NeRF-based methods on ScanNet and ARKitScenes datasets.

Achieves high detection accuracy with low computational overhead.

Demonstrates robustness across multiple indoor datasets.

Abstract

The key challenge of multi-view indoor 3D object detection is to infer accurate geometry information from images for precise 3D detection. Previous method relies on NeRF for geometry reasoning. However, the geometry extracted from NeRF is generally inaccurate, which leads to sub-optimal detection performance. In this paper, we propose MVSDet which utilizes plane sweep for geometry-aware 3D object detection. To circumvent the requirement for a large number of depth planes for accurate depth prediction, we design a probabilistic sampling and soft weighting mechanism to decide the placement of pixel features on the 3D volume. We select multiple locations that score top in the probability volume for each pixel and use their probability score to indicate the confidence. We further apply recent pixel-aligned Gaussian Splatting to regularize depth prediction and improve detection performance with little computation overhead. Extensive experiments on ScanNet and ARKitScenes datasets are conducted to show the superiority of our model. Our code is available at https://github.com/Pixie8888/MVSDet.