Non-asymptotic information theoretic bound for some multi-party scenarios

TL;DR

This paper derives non-asymptotic, one-shot information-theoretic bounds for distributed source coding and multiple-access channels, introducing a novel typical set based on smooth entropies that aligns with classical asymptotic results.

Contribution

It presents the first one-shot rate regions for these scenarios using a new typical set, bridging non-asymptotic and asymptotic analyses.

Findings

Achieves asymptotically optimal rate regions in the one-shot setting.

Introduces a novel one-shot typical set based on smooth entropies.

Results match classical asymptotic rate regions in the limit of many uses.

Abstract

In the last few years, there has been a great interest in extending the information-theoretic scenario for the non-asymptotic or one-shot case, i.e., where the channel is used only once. We provide the one-shot rate region for the distributed source-coding (Slepian-Wolf) and the multiple-access channel. Our results are based on defining a novel one-shot typical set based on smooth entropies that yields the one-shot achievable rate regions while leveraging the results from the asymptotic analysis. Our results are asymptotically optimal, i.e., for the distributed source coding they yield the same rate region as the Slepian-Wolf in the limit of unlimited independent and identically distributed (i.i.d.) copies. Similarly for the multiple-access channel the asymptotic analysis of our approach yields the rate region which is equal to the rate region of the memoryless multiple-access channel in the limit of large number of channel uses.