2.5D Object Detection for Intelligent Roadside Infrastructure

TL;DR

This paper presents a novel 2.5D object detection method for roadside infrastructure cameras, improving detection accuracy and robustness across viewpoints and weather conditions, aiding autonomous vehicle perception.

Contribution

The paper introduces a 2.5D detection framework that predicts ground plane parallelograms, enhancing generalization and robustness for infrastructure-mounted cameras compared to traditional 3D methods.

Findings

High detection accuracy across diverse conditions

Strong cross-viewpoint generalization

Robustness to weather and lighting variations

Abstract

On-board sensors of autonomous vehicles can be obstructed, occluded, or limited by restricted fields of view, complicating downstream driving decisions. Intelligent roadside infrastructure perception systems, installed at elevated vantage points, can provide wide, unobstructed intersection coverage, supplying a complementary information stream to autonomous vehicles via vehicle-to-everything (V2X) communication. However, conventional 3D object-detection algorithms struggle to generalize under the domain shift introduced by top-down perspectives and steep camera angles. We introduce a 2.5D object detection framework, tailored specifically for infrastructure roadside-mounted cameras. Unlike conventional 2D or 3D object detection, we employ a prediction approach to detect ground planes of vehicles as parallelograms in the image frame. The parallelogram preserves the planar position, size, and orientation of objects while omitting their height, which is unnecessary for most downstream applications. For training, a mix of real-world and synthetically generated scenes is leveraged. We evaluate generalizability on a held-out camera viewpoint and in adverse-weather scenarios absent from the training set. Our results show high detection accuracy, strong cross-viewpoint generalization, and robustness to diverse lighting and weather conditions. Model weights and inference code are provided at: https://gitlab.kit.edu/kit/aifb/ATKS/public/digit4taf/2.5d-object-detection