Towards Ubiquitous Indoor Positioning: Comparing Systems across Heterogeneous Datasets

TL;DR

This paper introduces a method for evaluating Indoor Positioning Systems across diverse datasets and scenarios, addressing the limitations of traditional local experiments and enabling more generalizable assessments.

Contribution

It proposes a novel approach to evaluate IPSs using multiple heterogeneous datasets, validated through three practical use cases.

Findings

Aggregation of evaluation metrics aids high-level comparison of IPSs

The approach demonstrates consistent performance across different datasets

Evaluation across multiple scenarios reveals system robustness and limitations

Abstract

The evaluation of Indoor Positioning Systems (IPS) mostly relies on local deployments in the researchers' or partners' facilities. The complexity of preparing comprehensive experiments, collecting data, and considering multiple scenarios usually limits the evaluation area and, therefore, the assessment of the proposed systems. The requirements and features of controlled experiments cannot be generalized since the use of the same sensors or anchors density cannot be guaranteed. The dawn of datasets is pushing IPS evaluation to a similar level as machine-learning models, where new proposals are evaluated over many heterogeneous datasets. This paper proposes a way to evaluate IPSs in multiple scenarios, that is validated with three use cases. The results prove that the proposed aggregation of the evaluation metric values is a useful tool for high-level comparison of IPSs.