HyperDet3D: Learning a Scene-conditioned 3D Object Detector

TL;DR

HyperDet3D introduces a scene-conditioned hypernetwork approach for 3D object detection, leveraging scene prior knowledge to improve accuracy and adaptability across different datasets.

Contribution

The paper proposes HyperDet3D, a novel scene-conditioned hypernetwork that learns scene-specific priors and adapts detection models dynamically at test time.

Findings

Achieves state-of-the-art results on ScanNet and SUN RGB-D datasets.

Effectively transfers scene knowledge across different datasets with domain gaps.

Utilizes a Multi-head Scene-specific Attention module for dynamic parameter control.

Abstract

A bathtub in a library, a sink in an office, a bed in a laundry room -- the counter-intuition suggests that scene provides important prior knowledge for 3D object detection, which instructs to eliminate the ambiguous detection of similar objects. In this paper, we propose HyperDet3D to explore scene-conditioned prior knowledge for 3D object detection. Existing methods strive for better representation of local elements and their relations without scene-conditioned knowledge, which may cause ambiguity merely based on the understanding of individual points and object candidates. Instead, HyperDet3D simultaneously learns scene-agnostic embeddings and scene-specific knowledge through scene-conditioned hypernetworks. More specifically, our HyperDet3D not only explores the sharable abstracts from various 3D scenes, but also adapts the detector to the given scene at test time. We propose a discriminative Multi-head Scene-specific Attention (MSA) module to dynamically control the layer parameters of the detector conditioned on the fusion of scene-conditioned knowledge. Our HyperDet3D achieves state-of-the-art results on the 3D object detection benchmark of the ScanNet and SUN RGB-D datasets. Moreover, through cross-dataset evaluation, we show the acquired scene-conditioned prior knowledge still takes effect when facing 3D scenes with domain gap.