A Dataset Similarity Evaluation Framework for Wireless Communications and Sensing

TL;DR

This paper presents a new framework for evaluating dataset similarity in wireless communications and sensing, aiding data augmentation, benchmarking, and model adaptation with high correlation to model performance.

Contribution

It introduces a model-agnostic, task-specific dataset similarity evaluation framework using topology-preserving dimensionality reduction and novel metrics.

Findings

Metrics show over 0.85 correlation with model performance.

Designed metrics outperform traditional dataset comparison methods.

Framework aids in dataset quality assessment and model retraining decisions.

Abstract

This paper introduces a task-specific, model-agnostic framework for evaluating dataset similarity, providing a means to assess and compare dataset realism and quality. Such a framework is crucial for augmenting real-world data, improving benchmarking, and making informed retraining decisions when adapting to new deployment settings, such as different sites or frequency bands. The proposed framework is employed to design metrics based on UMAP topology-preserving dimensionality reduction, leveraging Wasserstein and Euclidean distances on latent space KNN clusters. The designed metrics show correlations above 0.85 between dataset distances and model performances on a channel state information compression unsupervised machine learning task leveraging autoencoder architectures. The results show that the designed metrics outperform traditional methods.