Real-time Detection, Tracking, and Classification of Moving and Stationary Objects using Multiple Fisheye Images

TL;DR

This paper presents a real-time surround view system using fisheye cameras and deep neural networks to detect, track, and classify moving and stationary objects around an autonomous vehicle, demonstrating practical urban deployment.

Contribution

It introduces a minimal-overhead algorithm that merges fisheye camera views for real-time object detection and classification in autonomous vehicles.

Findings

Effective real-time detection and tracking of objects in urban environments.

Successful deployment on a real vehicle demonstrating practical feasibility.

Deep neural network accurately classifies moving objects in merged fisheye images.

Abstract

The ability to detect pedestrians and other moving objects is crucial for an autonomous vehicle. This must be done in real-time with minimum system overhead. This paper discusses the implementation of a surround view system to identify moving as well as static objects that are close to the ego vehicle. The algorithm works on 4 views captured by fisheye cameras which are merged into a single frame. The moving object detection and tracking solution uses minimal system overhead to isolate regions of interest (ROIs) containing moving objects. These ROIs are then analyzed using a deep neural network (DNN) to categorize the moving object. With deployment and testing on a real car in urban environments, we have demonstrated the practical feasibility of the solution. The video demos of our algorithm have been uploaded to Youtube: https://youtu.be/vpoCfC724iA, https://youtu.be/2X4aqH2bMBs