Task-Oriented Semantic Communication for Stereo-Vision 3D Object Detection

TL;DR

This paper introduces a semantic communication framework driven by optical flow for stereo-vision 3D object detection, significantly reducing data transmission while maintaining high detection accuracy in resource-constrained environments.

Contribution

It presents a novel optical flow-driven semantic communication method that prioritizes semantic information transmission for stereo-vision 3D detection, improving efficiency and accuracy.

Findings

Detection accuracy improved by nearly 70%.

Outperforms traditional methods in low SNR conditions.

Reduces data transmission without sacrificing detection performance.

Abstract

With the development of computer vision, 3D object detection has become increasingly important in many real-world applications. Limited by the computing power of sensor-side hardware, the detection task is sometimes deployed on remote computing devices or the cloud to execute complex algorithms, which brings massive data transmission overhead. In response, this paper proposes an optical flow-driven semantic communication framework for the stereo-vision 3D object detection task. The proposed framework fully exploits the dependence of stereo-vision 3D detection on semantic information in images and prioritizes the transmission of this semantic information to reduce total transmission data sizes while ensuring the detection accuracy. Specifically, we develop an optical flow-driven module to jointly extract and recover semantics from the left and right images to reduce the loss of the left-right photometric alignment semantic information and improve the accuracy of depth inference. Then, we design a 2D semantic extraction module to identify and extract semantic meaning around the objects to enhance the transmission of semantic information in the key areas. Finally, a fusion network is used to fuse the recovered semantics, and reconstruct the stereo-vision images for 3D detection. Simulation results show that the proposed method improves the detection accuracy by nearly 70% and outperforms the traditional method, especially for the low signal-to-noise ratio regime.