Boosting Real-Time Driving Scene Parsing with Shared Semantics

TL;DR

This paper introduces a dual-camera scene parsing framework that shares and fuses semantics to improve real-time autonomous driving perception, achieving higher accuracy with similar computational costs.

Contribution

It proposes a novel semantic sharing and fusion framework for dual-camera systems, enhancing parsing accuracy in real-time driving scenarios.

Findings

Outperforms baseline in parsing accuracy

Maintains comparable computational efficiency

Effective semantic sharing between cameras

Abstract

Real-time scene parsing is a fundamental feature for autonomous driving vehicles with multiple cameras. In this letter we demonstrate that sharing semantics between cameras with different perspectives and overlapped views can boost the parsing performance when compared with traditional methods, which individually process the frames from each camera. Our framework is based on a deep neural network for semantic segmentation but with two kinds of additional modules for sharing and fusing semantics. On the one hand, a semantics sharing module is designed to establish the pixel-wise mapping between the input images. Features as well as semantics are shared by the map to reduce duplicated workload which leads to more efficient computation. On the other hand, feature fusion modules are designed to combine different modal of semantic features, which leverage the information from both inputs for better accuracy. To evaluate the effectiveness of the proposed framework, we have applied our network to a dual-camera vision system for driving scene parsing. Experimental results show that our network outperforms the baseline method on the parsing accuracy with comparable computations.