Hyper-VIB: A Hypernetwork-Enhanced Information Bottleneck Approach for Task-Oriented Communications

TL;DR

Hyper-VIB introduces a hypernetwork-enhanced information bottleneck method for efficient task-oriented communication in 6G systems, achieving optimal accuracy and minimal communication with reduced training complexity.

Contribution

The paper proposes Hyper-VIB, a novel hypernetwork-based IB approach that efficiently optimizes task accuracy and communication overhead in 6G systems, with a tractable approximation and single-phase training.

Findings

Hyper-VIB outperforms conventional VIB in accuracy.

Hyper-VIB reduces training time significantly.

Theoretical validation confirms the hypernetwork design.

Abstract

This paper presents Hyper-VIB, a hypernetwork-enhanced information bottleneck (IB) approach designed to enable efficient task-oriented communications in 6G collaborative intelligent systems. Leveraging IB theory, our approach enables an optimal end-to-end joint training of device and network models, in terms of the maximal task execution accuracy as well as the minimal communication overhead, through optimizing the trade-off hyperparameter. To address computational intractability in high-dimensional IB optimization, a tractable variational upper-bound approximation is derived. Unlike conventional grid or random search methods that require multiple training rounds with substantial computational costs, Hyper-VIB introduces a hypernetwork that generates approximately optimal DNN parameters for different values of the hyperparameter within a single training phase. Theoretical analysis in the linear case validates the hypernetwork design. Experimental results demonstrate our Hyper-VIB's superior accuracy and training efficiency over conventional VIB approaches in both classification and regression tasks.