Towards Native Intelligence: 6G-LLM Trained with Reinforcement Learning from NDT Feedback

TL;DR

This paper introduces a novel reinforcement learning framework using digital twin feedback to train 6G domain large language models, enhancing network orchestration and adaptability beyond traditional offline methods.

Contribution

The paper proposes RLDTF, a new training paradigm for 6G-LLMs utilizing network digital twins and reinforcement learning for continual self-improvement.

Findings

Significant improvement in orchestration accuracy.

Enhanced solution optimality over state-of-the-art baselines.

Effective use of digital twin feedback for training.

Abstract

Owing to its comprehensive understanding of upper-layer application requirements and the capabilities of practical communication systems, the 6G-LLM (6G domain large language model) offers a promising pathway toward realizing network native intelligence. Serving as the system orchestrator, the 6G-LLM drives a paradigm shift that fundamentally departs from existing rule-based approaches, which primarily rely on modular, experience-driven optimization. By contrast, the 6G-LLM substantially enhances network flexibility and adaptability. Nevertheless, current efforts to construct 6G-LLMs are constrained by their reliance on large-scale, meticulously curated, human-authored corpora, which are impractical to obtain in real-world scenarios. Moreover, purely offline-trained models lack the capacity for continual self-improvement, limiting their ability to adapt to the highly dynamic requirements of wireless communication environments. To overcome these limitations, we propose a novel training paradigm termed RLDTF (Reinforcement Learning from Digital Twin Feedback) for 6G-LLMs. This framework leverages network digital twins to generate reward signals based on orchestration outcomes, while employing reinforcement learning to guide the model toward optimal decision-making dynamically. Furthermore, we introduce a weighted token mechanism to improve output accuracy. Comprehensive experimental results demonstrate that our proposed framework significantly outperforms state-of-the-art baselines in orchestration accuracy and solution optimality.