RBF Weighted Hyper-Involution for RGB-D Object Detection

TL;DR

This paper introduces a real-time RGB-D object detection model that effectively fuses depth and color features using novel RBF weighted hyper-involution and trainable fusion layers, improving performance on standard benchmarks.

Contribution

It proposes a novel RBF weighted hyper-involution for depth feature extraction and a trainable fusion layer for better RGB-D feature integration, enabling real-time performance.

Findings

Achieves state-of-the-art results on NYU Depth V2

Maintains competitive performance on SUN RGB-D

Demonstrates effective depth and color feature fusion

Abstract

A vast majority of augmented reality devices come equipped with depth and color cameras. Despite their advantages, extracting both photometric and depth features simultaneously in real-time remains challenging due to inherent differences between depth and color images. Furthermore, standard convolution operations are insufficient for extracting information directly from raw depth images, leading to inefficient intermediate representations. To address these issues, we propose a real-time two-stream RGBD object detection model. Our model introduces two new components: a dynamic radial basis function (RBF) weighted depth-based hyper-involution that adjusts dynamically based on spatial interaction patterns in raw depth maps, and an up-sampling based trainable fusion layer that combines extracted depth and color image features without obstructing information transfer between them. Experimental results demonstrate that the proposed approach achieves the strongest performance among existing RGB-D 2D object detection methods on NYU Depth V2, while remaining competitive on the SUN RGB-D benchmark.