Task and Perception-aware Distributed Source Coding for Correlated Speech under Bandwidth-constrained Channels

TL;DR

This paper introduces a neural distributed source coding method for correlated speech in bandwidth-limited wireless AR/VR applications, improving quality and adaptability over existing autoencoder approaches.

Contribution

It proposes a novel NDPCA-based distributed coding algorithm with perception-aware loss, enabling dynamic bitrate adaptation and leveraging source correlations.

Findings

Significant PSNR improvements over naive autoencoders

Approaches theoretical upper bound in low-bandwidth scenarios

Provides a rate-distortion-perception trade-off curve

Abstract

Emerging wireless AR/VR applications require real-time transmission of correlated high-fidelity speech from multiple resource-constrained devices over unreliable, bandwidth-limited channels. Existing autoencoder-based speech source coding methods fail to address the combination of the following - (1) dynamic bitrate adaptation without retraining the model, (2) leveraging correlations among multiple speech sources, and (3) balancing downstream task loss with realism of reconstructed speech. We propose a neural distributed principal component analysis (NDPCA)-aided distributed source coding algorithm for correlated speech sources transmitting to a central receiver. Our method includes a perception-aware downstream task loss function that balances perceptual realism with task-specific performance. Experiments show significant PSNR improvements under bandwidth constraints over naive autoencoder methods in task-agnostic (19%) and task-aware settings (52%). It also approaches the theoretical upper bound, where all correlated sources are sent to a single encoder, especially in low-bandwidth scenarios. Additionally, we present a rate-distortion-perception trade-off curve, enabling adaptive decisions based on application-specific realism needs.