Comparing Video Based Shoulder Surfing with Live Simulation

TL;DR

This study compares video-based shoulder surfing simulations with live experiments, finding that video simulations are generally reliable for Android graphical patterns but may underestimate attacker success in live scenarios, especially for PINs.

Contribution

It provides empirical evidence on the validity of video simulations as a baseline for shoulder surfing attacks compared to live experiments.

Findings

Video simulations match live results for Android patterns.

Live attackers outperform video simulations for PINs by up to 1.9x.

Removing feedback lines reduces security in live settings, but less so in video simulations.

Abstract

We analyze the claims that video recreations of shoulder surfing attacks offer a suitable alternative and a baseline, as compared to evaluation in a live setting. We recreated a subset of the factors of a prior video-simulation experiment conducted by Aviv et al. (ACSAC 2017), and model the same scenario using live participants ($n=36$) instead (i.e., the victim and attacker were both present). The live experiment confirmed that for Android's graphical patterns video simulation is consistent with the live setting for attacker success rates. However, both 4- and 6-digit PINs demonstrate statistically significant differences in attacker performance, with live attackers performing as much 1.9x better than in the video simulation. The security benefits gained from removing feedback lines in Android's graphical patterns are also greatly diminished in the live setting, particularly under multiple attacker observations, but overall, the data suggests that video recreations can provide a suitable baseline measure for attacker success rate. However, we caution that researchers should consider that these baselines may greatly underestimate the threat of an attacker in live settings.