Zero-Interaction Security -- Towards Sound Experimental Validation

TL;DR

This paper discusses the importance of reproducibility and realistic datasets in zero-interaction security research, sharing experiences in reproducing schemes, collecting real-world data, and evaluating results to improve scientific rigor.

Contribution

It provides a comprehensive case study on reproducing ZIS schemes, collecting real-world data, and emphasizes the need for reproducibility and realistic datasets in security research.

Findings

Reproduced five state-of-the-art ZIS schemes

Collected and released a real-world sensor dataset

Identified technical and methodological challenges

Abstract

Reproducibility and realistic datasets are crucial for advancing research. Unfortunately, they are often neglected as valid scientific contributions in many young disciplines, with computer science being no exception. In this article, we show the challenges encountered when reproducing the work of others, collecting realistic data in the wild, and ensuring that our own work is reproducible in turn. The presented findings are based on our study investigating the limits of zero-interaction security (ZIS) -- a novel concept, leveraging sensor data collected by Internet of Things (IoT) devices to pair or authenticate devices. In particular, we share our experiences in reproducing five state-of-the-art ZIS schemes, collecting a comprehensive dataset of sensor data from the real world, evaluating these schemes on the collected data, and releasing the data, code, and documentation to facilitate reproducibility of our results. In our discussion, we outline general considerations when conducting similar studies and give specific examples of technical and methodological issues that we experienced. We hope that our findings will raise awareness about the importance of reproducibility and realistic datasets in computer science and inform future research.