Hyperbolic Distillation: Geometry-Guided Cross-Modal Transfer for Robust 3D Object Detection

TL;DR

This paper introduces HGC-Det, a hyperbolic geometry-guided cross-modal distillation framework that enhances 3D object detection by effectively fusing point cloud and image features.

Contribution

The paper proposes a novel hyperbolic constrained distillation method with components for semantic-guided voxel optimization, hyperbolic feature transfer, and geometry-based feature aggregation.

Findings

Achieves improved detection accuracy on SUN RGB-D, ARKitScenes, KITTI, and nuScenes datasets.

Effectively mitigates semantic loss during feature fusion.

Balances detection performance with computational efficiency.

Abstract

Cross-modal knowledge distillation has emerged as an effective strategy for integrating point cloud and image features in 3D perception tasks. However, the modality heterogeneity, spatial misalignment, and the representation crisis of multiple modalities often limit the efficient of these cross-modal distillation methods. To address these limitations in existing approaches, we propose a hyperbolic constrained cross-modal distillation method for multimodal 3D object detection (HGC-Det). The proposed HGC-Det framework includes an image branch and a point cloud branch to extract semantic features from two different modalities. The point cloud branch comprises three core components: a 2D semantic-guided voxel optimization component (SGVO), a hyperbolic geometry constrained cross-modal feature transfer component (HFT), and a feature aggregation-based geometry optimization component (FAGO). Specifically, the SGVO component adaptively refines the spatial representation of the 3D branch by leveraging semantic cues from the image branch, thereby mitigating the issue of inadequate representation fusion. The HFT component exploits the intrinsic geometric properties of hyperbolic space to alleviate semantic loss during the fusion of high-dimensional image features and low-dimensional point cloud features. Finally, the FAGO compensates for potential spatial feature degradation introduced by the 2D semantic-guided voxel optimization component. Extensive experiments on indoor datasets (SUN RGB-D, ARKitScenes) and outdoor datasets (KITTI, nuScenes) demonstrate that our method achieves a better trade-off between detection accuracy and computational cost.