Multimodal Trustworthy Semantic Communication for Audio-Visual Event Localization

TL;DR

This paper presents MMTrustSC, a novel framework that enhances the security and reliability of multimodal semantic communication for audio-visual event localization, using advanced encoding and encryption techniques validated through simulations.

Contribution

Introduction of MMTrustSC, a comprehensive framework combining semantic encoding, error correction, and hybrid encryption to improve secure multimodal data transmission for AVE localization.

Findings

Significant improvements in data accuracy and reliability.

Effective safeguarding of semantic data integrity and privacy.

Enhanced system performance in noisy environments.

Abstract

The exponential growth in wireless data traffic, driven by the proliferation of mobile devices and smart applications, poses significant challenges for modern communication systems. Ensuring the secure and reliable transmission of multimodal semantic information is increasingly critical, particularly for tasks like Audio-Visual Event (AVE) localization. This letter introduces MMTrustSC, a novel framework designed to address these challenges by enhancing the security and reliability of multimodal communication. MMTrustSC incorporates advanced semantic encoding techniques to safeguard data integrity and privacy. It features a two-level coding scheme that combines error-correcting codes with conventional encoders to improve the accuracy and reliability of multimodal data transmission. Additionally, MMTrustSC employs hybrid encryption, integrating both asymmetric and symmetric encryption methods, to secure semantic information and ensure its confidentiality and integrity across potentially hostile networks. Simulation results validate MMTrustSC's effectiveness, demonstrating substantial improvements in data transmission accuracy and reliability for AVE localization tasks. This framework represents a significant advancement in managing intermodal information complementarity and mitigating physical noise, thus enhancing overall system performance.