Spatial-Temporal Graph Enhanced DETR Towards Multi-Frame 3D Object Detection

TL;DR

STEMD introduces a novel framework that enhances multi-frame 3D object detection by integrating spatial-temporal graph attention, previous frame outputs, and IoU regularization to improve accuracy and handle complex scenarios.

Contribution

The paper proposes a new end-to-end framework that combines graph attention, temporal information, and query regularization for improved multi-frame 3D detection.

Findings

Effective modeling of object interactions with graph attention.

Improved detection accuracy in challenging scenarios.

Minor computational overhead compared to existing methods.

Abstract

The Detection Transformer (DETR) has revolutionized the design of CNN-based object detection systems, showcasing impressive performance. However, its potential in the domain of multi-frame 3D object detection remains largely unexplored. In this paper, we present STEMD, a novel end-to-end framework that enhances the DETR-like paradigm for multi-frame 3D object detection by addressing three key aspects specifically tailored for this task. First, to model the inter-object spatial interaction and complex temporal dependencies, we introduce the spatial-temporal graph attention network, which represents queries as nodes in a graph and enables effective modeling of object interactions within a social context. To solve the problem of missing hard cases in the proposed output of the encoder in the current frame, we incorporate the output of the previous frame to initialize the query input of the decoder. Finally, it poses a challenge for the network to distinguish between the positive query and other highly similar queries that are not the best match. And similar queries are insufficiently suppressed and turn into redundant prediction boxes. To address this issue, our proposed IoU regularization term encourages similar queries to be distinct during the refinement. Through extensive experiments, we demonstrate the effectiveness of our approach in handling challenging scenarios, while incurring only a minor additional computational overhead. The code is publicly available at https://github.com/Eaphan/STEMD.