MonoVQD: Monocular 3D Object Detection with Variational Query Denoising and Self-Distillation

TL;DR

MonoVQD introduces a novel framework for monocular 3D object detection that enhances DETR-based architectures through variational query denoising, self-distillation, and a new self-attention mechanism, achieving superior results on benchmarks.

Contribution

The paper presents three innovations: a mask separated self-attention mechanism, variational query denoising to address gradient issues, and a self-distillation strategy, advancing DETR-based monocular 3D detection.

Findings

Achieves state-of-the-art performance on KITTI benchmark.

Enhances detection accuracy in multi-view 3D scenarios.

Demonstrates broad applicability and robustness of the proposed methods.

Abstract

Precisely localizing 3D objects from a single image constitutes a central challenge in monocular 3D detection. While DETR-like architectures offer a powerful paradigm, their direct application in this domain encounters inherent limitations, preventing optimal performance. Our work addresses these challenges by introducing MonoVQD, a novel framework designed to fundamentally advance DETR-based monocular 3D detection. We propose three main contributions. First, we propose the Mask Separated Self-Attention mechanism that enables the integration of the denoising process into a DETR architecture. This improves the stability of Hungarian matching to achieve a consistent optimization objective. Second, we present the Variational Query Denoising technique to address the gradient vanishing problem of conventional denoising methods, which severely restricts the efficiency of the denoising process. This explicitly introduces stochastic properties to mitigate this fundamental limitation and unlock substantial performance gains. Finally, we introduce a sophisticated self-distillation strategy, leveraging insights from later decoder layers to synergistically improve query quality in earlier layers, thereby amplifying the iterative refinement process. Rigorous experimentation demonstrates that MonoVQD achieves superior performance on the challenging KITTI monocular benchmark. Highlighting its broad applicability, MonoVQD's core components seamlessly integrate into other architectures, delivering significant performance gains even in multi-view 3D detection scenarios on the nuScenes dataset and underscoring its robust generalization capabilities.