Learned Two-Plane Perspective Prior based Image Resampling for Efficient Object Detection

TL;DR

This paper introduces a learnable, geometry-guided image resampling method that significantly enhances small and distant object detection in real-time autonomous perception systems, outperforming state-of-the-art approaches.

Contribution

It proposes a novel geometry-aware prior for image resampling that improves detection accuracy and efficiency, especially for small and far-away objects, in real-time perception tasks.

Findings

Improves small object detection by +4.1 AP_S and +39% in autonomous navigation.

Enhances real-time detection performance by +5.3 sAP_S or +63%.

Detects small objects at scales other methods cannot, with significant performance gains.

Abstract

Real-time efficient perception is critical for autonomous navigation and city scale sensing. Orthogonal to architectural improvements, streaming perception approaches have exploited adaptive sampling improving real-time detection performance. In this work, we propose a learnable geometry-guided prior that incorporates rough geometry of the 3D scene (a ground plane and a plane above) to resample images for efficient object detection. This significantly improves small and far-away object detection performance while also being more efficient both in terms of latency and memory. For autonomous navigation, using the same detector and scale, our approach improves detection rate by +4.1 $AP_{S}$ or +39% and in real-time performance by +5.3 $sAP_{S}$ or +63% for small objects over state-of-the-art (SOTA). For fixed traffic cameras, our approach detects small objects at image scales other methods cannot. At the same scale, our approach improves detection of small objects by 195% (+12.5 $AP_{S}$) over naive-downsampling and 63% (+4.2 $AP_{S}$) over SOTA.