Computing Capacity-Cost Functions for Continuous Channels in Wasserstein Space

TL;DR

This paper introduces a novel iterative algorithm using Wasserstein distance for computing capacity-cost functions of continuous channels, unifying approaches for rate-distortion and capacity problems with practical particle-based implementation.

Contribution

It proposes a Wasserstein distance-based proximal reformulation of the Blahut-Arimoto algorithm for continuous channels, enabling practical particle-based computation of capacity-cost and rate-distortion functions.

Findings

Effective particle-based implementation using importance sampling.

Unified framework for capacity-cost and rate-distortion functions.

Algorithm applicable to continuous source and channel spaces.

Abstract

This paper investigates the problem of computing capacity-cost (C-C) functions for continuous channels. Motivated by the Kullback-Leibler divergence (KLD) proximal reformulation of the classical Blahut-Arimoto (BA) algorithm, the Wasserstein distance is introduced to the proximal term for the continuous case, resulting in an iterative algorithm related to the Wasserstein gradient descent. Practical implementation involves moving particles along the negative gradient direction of the objective function's first variation in the Wasserstein space and approximating integrals by the importance sampling (IS) technique. Such formulation is also applied to the rate-distortion (R-D) function for continuous source spaces and thus provides a unified computation framework for both problems.