MC-BEVRO: Multi-Camera Bird Eye View Road Occupancy Detection for Traffic Monitoring

TL;DR

This paper presents a multi-camera Bird's-Eye-View framework for road occupancy detection in traffic monitoring, addressing occlusion and view angle challenges through novel fusion methods and synthetic data, with promising real-world generalization.

Contribution

Introduces a multi-camera BEV perception framework with novel fusion techniques and synthetic dataset, demonstrating effective sim-to-real transfer for traffic monitoring applications.

Findings

Multi-camera fusion improves detection accuracy.

Synthetic dataset enables robust model training.

Zero-shot and few-shot transfer show good real-world generalization.

Abstract

Single camera 3D perception for traffic monitoring faces significant challenges due to occlusion and limited field of view. Moreover, fusing information from multiple cameras at the image feature level is difficult because of different view angles. Further, the necessity for practical implementation and compatibility with existing traffic infrastructure compounds these challenges. To address these issues, this paper introduces a novel Bird's-Eye-View road occupancy detection framework that leverages multiple roadside cameras to overcome the aforementioned limitations. To facilitate the framework's development and evaluation, a synthetic dataset featuring diverse scenes and varying camera configurations is generated using the CARLA simulator. A late fusion and three early fusion methods were implemented within the proposed framework, with performance further enhanced by integrating backgrounds. Extensive evaluations were conducted to analyze the impact of multi-camera inputs and varying BEV occupancy map sizes on model performance. Additionally, a real-world data collection pipeline was developed to assess the model's ability to generalize to real-world environments. The sim-to-real capabilities of the model were evaluated using zero-shot and few-shot fine-tuning, demonstrating its potential for practical application. This research aims to advance perception systems in traffic monitoring, contributing to improved traffic management, operational efficiency, and road safety.