A Flexible Multi-view Multi-modal Imaging System for Outdoor Scenes

TL;DR

This paper introduces a novel wireless multi-view multi-modal imaging system for large outdoor scenes, integrating cameras and LiDARs to enhance 3D object detection and tracking accuracy in complex environments.

Contribution

It presents the first large outdoor multi-view multi-modal 3D imaging system with automatic calibration, scalable wireless sensor network, and demonstrates improved detection and tracking performance.

Findings

Multi-view point clouds significantly improve detection accuracy.

The system effectively operates in complex outdoor environments.

Automatic spatio-temporal calibration ensures accurate data collection.

Abstract

Multi-view imaging systems enable uniform coverage of 3D space and reduce the impact of occlusion, which is beneficial for 3D object detection and tracking accuracy. However, existing imaging systems built with multi-view cameras or depth sensors are limited by the small applicable scene and complicated composition. In this paper, we propose a wireless multi-view multi-modal 3D imaging system generally applicable to large outdoor scenes, which consists of a master node and several slave nodes. Multiple spatially distributed slave nodes equipped with cameras and LiDARs are connected to form a wireless sensor network. While providing flexibility and scalability, the system applies automatic spatio-temporal calibration techniques to obtain accurate 3D multi-view multi-modal data. This system is the first imaging system that integrates mutli-view RGB cameras and LiDARs in large outdoor scenes among existing 3D imaging systems. We perform point clouds based 3D object detection and long-term tracking using the 3D imaging dataset collected by this system. The experimental results show that multi-view point clouds greatly improve 3D object detection and tracking accuracy regardless of complex and various outdoor environments.