High and Low Resolution Tradeoffs in Roadside Multimodal Sensing

TL;DR

This paper introduces a simulation and evaluation framework for roadside sensing systems, demonstrating that combining low-resolution radar with low-res LiDAR can achieve significant performance gains at lower costs, challenging the assumption that higher resolution always yields better results.

Contribution

The paper presents a novel ex-ante evaluation framework and simulation tool for sensor placement and modality comparison, emphasizing multimodal fusion benefits.

Findings

Fusing radar with low-res LiDAR improves pedestrian detection by 14% AP.

The combined sensor setup outperforms high-res LiDAR alone in cost-effectiveness.

The framework is robust across different neural network architectures.

Abstract

Balancing cost and performance is crucial when choosing high- versus low-resolution point-cloud roadside sensors. For example, LiDAR delivers dense point cloud, while 4D millimeter-wave radar, though spatially sparser, embeds velocity cues that help distinguish objects and come at a lower price. Unfortunately, the sensor placement strategies will influence point cloud density and distribution across the coverage area. Compounding the first challenge is the fact that different sensor mixtures often demand distinct neural network architectures to maximize their complementary strengths. Without an evaluation framework that establishes a benchmark for comparison, it is imprudent to make claims regarding whether marginal gains result from higher resolution and new sensing modalities or from the algorithms. We present an ex-ante evaluation that addresses the two challenges. First, we realized a simulation tool that builds on integer programming to automatically compare different sensor placement strategies against coverage and cost jointly. Additionally, inspired by human multi-sensory integration, we propose a modular framework to assess whether reductions in spatial resolution can be compensated by informational richness in detecting traffic participants. Extensive experimental testing on the proposed framework shows that fusing velocity-encoded radar with low-resolution LiDAR yields marked gains (14 percent AP for pedestrians and an overall mAP improvement of 1.5 percent across six categories) at lower cost than high-resolution LiDAR alone. Notably, these marked gains hold regardless of the specific deep neural modules employed in our frame. The result challenges the prevailing assumption that high resolution are always superior to low-resolution alternatives.