Diffusion models for audio semantic communication

TL;DR

This paper introduces a robust audio semantic communication framework using diffusion models that transmits lower-dimensional representations, effectively restoring audio content and semantics even under noisy channel conditions.

Contribution

It proposes a novel generative framework that treats audio transmission as an inverse problem, improving robustness to noise and corruption through diffusion-based generation.

Findings

Outperforms existing methods in real-world noisy channels

Effectively restores corrupted and missing audio parts

Focuses on semantic content preservation during transmission

Abstract

Directly sending audio signals from a transmitter to a receiver across a noisy channel may absorb consistent bandwidth and be prone to errors when trying to recover the transmitted bits. On the contrary, the recent semantic communication approach proposes to send the semantics and then regenerate semantically consistent content at the receiver without exactly recovering the bitstream. In this paper, we propose a generative audio semantic communication framework that faces the communication problem as an inverse problem, therefore being robust to different corruptions. Our method transmits lower-dimensional representations of the audio signal and of the associated semantics to the receiver, which generates the corresponding signal with a particular focus on its meaning (i.e., the semantics) thanks to the conditional diffusion model at its core. During the generation process, the diffusion model restores the received information from multiple degradations at the same time including corruption noise and missing parts caused by the transmission over the noisy channel. We show that our framework outperforms competitors in a real-world scenario and with different channel conditions. Visit the project page to listen to samples and access the code: https://ispamm.github.io/diffusion-audio-semantic-communication/.