Toward Realistic Wi-Fi Fault Diagnosis: A Multi-Modal Benchmark

TL;DR

This paper introduces a comprehensive multi-modal Wi-Fi fault dataset from real-world environments, establishing a benchmark for diagnosis tasks and evaluating the effectiveness of current approaches including LLM-based methods.

Contribution

It provides one of the first publicly available multi-modal Wi-Fi fault datasets and a unified benchmark for diverse diagnosis tasks in heterogeneous environments.

Findings

Existing diagnosis approaches struggle to leverage heterogeneous data effectively.

LLM-based approaches show potential but require further evaluation.

The benchmark reveals key challenges and considerations for future Wi-Fi fault diagnosis.

Abstract

Intelligent network operation and maintenance systems in modern networks continuously generate large volumes of multi-modal operational data. However, Wi-Fi fault diagnosis under heterogeneous operational environments remains insufficiently understood. We build a real-world Wi-Fi testbed deployed in campus working environments with an automated fault injection system, and collect a multi-modal Wi-Fi fault dataset containing over 10,000 fault samples across diverse wireless scenarios. To the best of our knowledge, this is among the first publicly available datasets jointly capturing heterogeneous cross-layer operational observations for Wi-Fi fault diagnosis. Based on this dataset, we establish a unified benchmark spanning multiple diagnosis tasks, operational modalities, and representative diagnosis paradigms. Experimental results indicate that effectively leveraging heterogeneous operational data remains challenging for existing diagnosis approaches. We further evaluate emerging LLM-based approaches and develop a reasoningoriented evaluation framework to assess the consistency between generated diagnostic analyses and actual network conditions. Our findings suggest several important considerations for future multi-modal Wi-Fi diagnosis.