DEGround: An Effective Baseline for Ego-centric 3D Visual Grounding with a Homogeneous Framework

TL;DR

DEGround introduces a unified, object-centric transformer framework for ego-centric 3D visual grounding, enhancing performance and efficiency over traditional heterogeneous pipelines.

Contribution

It proposes a homogeneous detection and grounding framework with shared queries and introduces two modules for improved instruction grounding.

Findings

Achieves state-of-the-art results on multiple benchmarks.

Outperforms previous methods by 7.52% on EmbodiedScan.

Demonstrates the effectiveness of shared object representations.

Abstract

A core task in embodied intelligence is ego-centric 3D visual grounding. Existing methods typically adopt two-stage, heterogeneous pipelines that pair a detector with a separate grounding model. Incompatible decoders and box heads hinder the transfer of object-level priors, and the split training causes redundant re-optimization. To overcome these limitations, we present DEGround, a straight, elegant, and effective framework that centers on object-level sharing over detection and grounding. It employs a set of queries that serves as the common object representation for both detection and grounding, which is decoded by a shared transformer and bounding box head. Building on this homogeneous framework, we further introduce two task-specific plug-in modules to enhance fine-grained instruction grounding. The Regional Activation Grounding module improves spatial-textual alignment by highlighting instruction-relevant regions, while the Query-wise Modulation module applies sentence-conditioned affine modulation to generate instruction-aware queries at initialization. Extensive experiments demonstrate that DEGround achieves the best performance on multiple benchmarks. Remarkably, it significantly outperforms previous methods by 7.52% at overall precision on the EmbodiedScan dataset.