Importance-Aware Image Segmentation-based Semantic Communication for Autonomous Driving

TL;DR

This paper introduces VIS-SemCom, a semantic communication system for autonomous driving that emphasizes important objects in image segmentation to improve safety and efficiency, utilizing a Swin Transformer-based codec and importance-aware strategies.

Contribution

The paper presents a novel importance-aware semantic communication system with a Swin Transformer-based codec and multi-scale semantic extraction tailored for autonomous driving.

Findings

Achieves nearly 6 dB coding gain with 60% mIoU

Reduces data transmission by up to 70% with 60% mIoU

Improves important object segmentation IoU by 4%

Abstract

This article studies the problem of image segmentation-based semantic communication in autonomous driving. In real traffic scenes, detecting the key objects (e.g., vehicles, pedestrians and obstacles) is more crucial than that of other objects to guarantee driving safety. Therefore, we propose a vehicular image segmentation-oriented semantic communication system, termed VIS-SemCom, where image segmentation features of important objects are transmitted to reduce transmission redundancy. First, to accurately extract image semantics, we develop a semantic codec based on Swin Transformer architecture, which expands the perceptual field thus improving the segmentation accuracy. Next, we propose a multi-scale semantic extraction scheme via assigning the number of Swin Transformer blocks for diverse resolution features, thus highlighting the important objects' accuracy. Furthermore, the importance-aware loss is invoked to emphasize the important objects, and an online hard sample mining (OHEM) strategy is proposed to handle small sample issues in the dataset. Experimental results demonstrate that the proposed VIS-SemCom can achieve a coding gain of nearly 6 dB with a 60% mean intersection over union (mIoU), reduce the transmitted data amount by up to 70% with a 60% mIoU, and improve the segmentation intersection over union (IoU) of important objects by 4%, compared to traditional transmission scheme.