Validation of Practicality for CSI Sensing Utilizing Machine Learning

TL;DR

This paper evaluates the effectiveness of various machine learning models using CSI data for human posture recognition, revealing high accuracy in controlled environments but significant performance drops in new settings, emphasizing generalization challenges.

Contribution

It systematically compares five ML models for CSI-based posture recognition and assesses their spatial generalization, highlighting the limitations of current approaches.

Findings

Naive Bayes-Support Vector Machine and Deep Learning models achieve over 85% accuracy in original settings.

Model accuracy drops to around 30% in different environments, indicating poor spatial generalization.

Performance within a fixed environment does not reliably translate to new spatial conditions.

Abstract

In this study, we leveraged Channel State Information (CSI), commonly utilized in WLAN communication, as training data to develop and evaluate five distinct machine learning models for recognizing human postures: standing, sitting, and lying down. The models we employed were: (i) Linear Discriminant Analysis, (ii) Naive Bayes-Support Vector Machine, (iii) Kernel-Support Vector Machine, (iv) Random Forest, and (v) Deep Learning. We systematically analyzed how the accuracy of these models varied with different amounts of training data. Additionally, to assess their spatial generalization capabilities, we evaluated the models' performance in a setting distinct from the one used for data collection. The experimental findings indicated that while two models -- (ii) Naive Bayes-Support Vector Machine and (v) Deep Learning -- achieved 85% or more accuracy in the original setting, their accuracy dropped to approximately 30% when applied in a different environment. These results underscore that although CSI-based machine learning models can attain high accuracy within a consistent spatial structure, their performance diminishes considerably with changes in spatial conditions, highlighting a significant challenge in their generalization capabilities.