CONVERGE: A Multi-Agent Vision-Radio Architecture for xApps

TL;DR

This paper introduces CONVERGE, a multi-agent architecture integrating real-time vision and radio data to enhance 5G/6G network control through sensing, enabling obstacle detection and improved beamforming with low latency.

Contribution

It presents a novel multi-agent architecture and a new video function for real-time sensing data delivery to xApps, bridging telecommunications and computer vision.

Findings

Sensing information delay is under 1 ms.

xApps can effectively use radio and video data for real-time RAN control.

The approach enables integrated sensing and communications in 5G/6G networks.

Abstract

Telecommunications and computer vision have evolved independently. With the emergence of high-frequency wireless links operating mostly in line-of-sight, visual data can help predict the channel dynamics by detecting obstacles and help overcoming them through beamforming or handover techniques. This paper proposes a novel architecture for delivering real-time radio and video sensing information to O-RAN xApps through a multi-agent approach, and introduces a new video function capable of generating blockage information for xApps, enabling Integrated Sensing and Communications. Experimental results show that the delay of sensing information remains under 1\,ms and that an xApp can successfully use radio and video sensing information to control the 5G/6G RAN in real-time.