When Attention is Beneficial for Learning Wireless Resource Allocation Efficiently?

TL;DR

This paper investigates whether attention mechanisms are necessary in GNNs for wireless resource allocation, revealing that attention is crucial when interference isn't directly measurable, and proposes a framework for designing effective GNNs.

Contribution

The paper provides a theoretical analysis of permutation-equivariant functions, showing when attention mechanisms are needed in GNNs for resource allocation, and validates the approach with simulations.

Findings

Attention is essential when interference isn't reflected in measurable parameters.

Re-expressed algorithms highlight the role of attention in modeling interference.

Proposed GNN framework improves learning efficiency in RIS-aided hybrid precoding.

Abstract

Owing to the use of attention mechanism to leverage the dependency across tokens, Transformers are efficient for natural language processing. By harnessing permutation properties broadly exist in resource allocation policies, each mapping measurable environmental parameters (e.g., channel matrix) to optimized variables (e.g., precoding matrix), graph neural networks (GNNs) are promising for learning these policies efficiently in terms of scalability and generalizability. To reap the benefits of both architectures, there is a recent trend of incorporating attention mechanism with GNNs for learning wireless policies. Nevertheless, is the attention mechanism really needed for resource allocation? In this paper, we strive to answer this question by analyzing the structures of functions defined on sets and numerical algorithms, given that the permutation properties of wireless policies are induced by the involved sets (say user set). In particular, we prove that the permutation equivariant functions on a single set can be recursively expressed by two types of functions: one involves attention, and the other does not. We proceed to re-express the numerical algorithms for optimizing several representative resource allocation problems in recursive forms. We find that when interference (say multi-user or inter-data stream interference) is not reflected in the measurable parameters of a policy, attention needs to be used to model the interference. With the insight, we establish a framework of designing GNNs by aligning with the structures. By taking reconfigurable intelligent surface-aided hybrid precoding as an example, the learning efficiency of the proposed GNN is validated via simulations.