DiffRef3D: A Diffusion-based Proposal Refinement Framework for 3D Object Detection

TL;DR

DiffRef3D introduces a diffusion-based framework for refining 3D object proposals in point cloud data, significantly enhancing detection accuracy by iteratively denoising residuals during training and inference.

Contribution

It is the first to apply diffusion processes to 3D object proposal refinement, offering a versatile method that improves existing detection models.

Findings

Consistently improves 3D detection performance on KITTI benchmark

Effective in refining proposals through iterative denoising

Versatile framework applicable to various 3D detectors

Abstract

Denoising diffusion models show remarkable performances in generative tasks, and their potential applications in perception tasks are gaining interest. In this paper, we introduce a novel framework named DiffRef3D which adopts the diffusion process on 3D object detection with point clouds for the first time. Specifically, we formulate the proposal refinement stage of two-stage 3D object detectors as a conditional diffusion process. During training, DiffRef3D gradually adds noise to the residuals between proposals and target objects, then applies the noisy residuals to proposals to generate hypotheses. The refinement module utilizes these hypotheses to denoise the noisy residuals and generate accurate box predictions. In the inference phase, DiffRef3D generates initial hypotheses by sampling noise from a Gaussian distribution as residuals and refines the hypotheses through iterative steps. DiffRef3D is a versatile proposal refinement framework that consistently improves the performance of existing 3D object detection models. We demonstrate the significance of DiffRef3D through extensive experiments on the KITTI benchmark. Code will be available.