To Theoretically Understand Transformer-Based In-Context Learning for Optimizing CSMA

TL;DR

This paper introduces a transformer-based in-context learning approach to optimize CSMA channel access, achieving near-optimal throughput and fast convergence in dynamic environments, surpassing existing methods.

Contribution

It is the first to apply transformer-based ICL for CSMA optimization, enabling effective contention window prediction with limited data and robustness to erroneous inputs.

Findings

Achieves near-optimal contention window prediction.

Demonstrates fast convergence and high throughput in NS-3 simulations.

Outperforms existing model-based and DRL-based approaches.

Abstract

The binary exponential backoff scheme is widely used in WiFi 7 and still incurs poor throughput performance under dynamic channel environments. Recent model-based approaches (e.g., non-persistent and $p$-persistent CSMA) simply optimize backoff strategies under a known and fixed node density, still leading to a large throughput loss due to inaccurate node density estimation. This paper is the first to propose LLM transformer-based in-context learning (ICL) theory for optimizing channel access. We design a transformer-based ICL optimizer to pre-collect collision-threshold data examples and a query collision case. They are constructed as a prompt as the input for the transformer to learn the pattern, which then generates a predicted contention window threshold (CWT). To train the transformer for effective ICL, we develop an efficient algorithm and guarantee a near-optimal CWT prediction within limited training steps. As it may be hard to gather perfect data examples for ICL in practice, we further extend to allow erroneous data input in the prompt. We prove that our optimizer maintains minimal prediction and throughput deviations from the optimal values. Experimental results on NS-3 further demonstrate our approach's fast convergence and near-optimal throughput over existing model-based and DRL-based approaches under unknown node densities.