Data Fusion of Semantic and Depth Information in the Context of Object Detection

TL;DR

This paper presents a method combining semantic and depth data fusion using stereo vision and CNNs to improve object detection and distance estimation for autonomous driving safety.

Contribution

It introduces a novel fusion approach integrating semantic segmentation and depth information with CNN-based object classification for enhanced detection accuracy.

Findings

Achieved accurate 3D position estimation of pedestrians.

Enhanced object detection performance using data fusion.

Demonstrated effectiveness on customized autonomous driving datasets.

Abstract

Considerable study has already been conducted regarding autonomous driving in modern era. An autonomous driving system must be extremely good at detecting objects surrounding the car to ensure safety. In this paper, classification, and estimation of an object's (pedestrian) position (concerning an ego 3D coordinate system) are studied and the distance between the ego vehicle and the object in the context of autonomous driving is measured. To classify the object, faster Region-based Convolution Neural Network (R-CNN) with inception v2 is utilized. First, a network is trained with customized dataset to estimate the reference position of objects as well as the distance from the vehicle. From camera calibration to computing the distance, cutting-edge technologies of computer vision algorithms in a series of processes are applied to generate a 3D reference point of the region of interest. The foremost step in this process is generating a disparity map using the concept of stereo vision.