An intelligent modular real-time vision-based system for environment perception

TL;DR

This paper introduces a modular, real-time vision-based system for environment perception in vehicles, combining multiple perception modules with novel techniques, aiming for broad applicability with minimal hardware requirements.

Contribution

The paper presents a novel modular perception system with improved accuracy and real-time performance, suitable for diverse vehicles and environments.

Findings

System accuracy exceeds 80% in all modules

Achieves real-time inference suitable for vehicle deployment

Demonstrates effectiveness on public and local datasets

Abstract

A significant portion of driving hazards is caused by human error and disregard for local driving regulations; Consequently, an intelligent assistance system can be beneficial. This paper proposes a novel vision-based modular package to ensure drivers' safety by perceiving the environment. Each module is designed based on accuracy and inference time to deliver real-time performance. As a result, the proposed system can be implemented on a wide range of vehicles with minimum hardware requirements. Our modular package comprises four main sections: lane detection, object detection, segmentation, and monocular depth estimation. Each section is accompanied by novel techniques to improve the accuracy of others along with the entire system. Furthermore, a GUI is developed to display perceived information to the driver. In addition to using public datasets, like BDD100K, we have also collected and annotated a local dataset that we utilize to fine-tune and evaluate our system. We show that the accuracy of our system is above 80% in all the sections. Our code and data are available at https://github.com/Pandas-Team/Autonomous-Vehicle-Environment-Perception